agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH 4/4] Documentation for slot-limit feature
72+ messages / 10 participants
[nested] [flat]
* [PATCH 4/4] Documentation for slot-limit feature
@ 2018-01-11 06:00 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Kyotaro Horiguchi @ 2018-01-11 06:00 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 28 ++++++++++++++++++++++++++++
doc/src/sgml/config.sgml | 22 ++++++++++++++++++++++
doc/src/sgml/high-availability.sgml | 8 +++++---
3 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 07e8b3325f..7e31267d68 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9887,6 +9887,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</entry>
</row>
+ <row>
+ <entry><structfield>wal_status</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+
+ <entry>Availability of WAL records claimed by the
+ slot. <literal>streaming</literal>, <literal>keeping</literal>,
+ <literal>lost</literal>
+ or <literal>unknown</literal>. <literal>streaming</literal> means that
+ the claimed records are available. <literal>keeping</literal> means that
+ some of them are to be removed by the next checkpoint.
+ <literal>lost</literal> means that some of them have been removed. The
+ last two states are seen only when
+ <xref linkend="guc-max-slot-wal-keep-size"/> is not zero. If the slot
+ doesn't have valid restart_lsn, this field
+ is <literal>unknown</literal>.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>remain</structfield></entry>
+ <entry><type>bigint</type></entry>
+ <entry></entry>
+ <entry>The amount in bytes that WAL location (LSN) can advance until the
+ slot may lose required WAL records.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index bee4afbe4e..9d190c3daa 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3116,6 +3116,28 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-slot-wal-keep-size" xreflabel="max_slot_wal_keep_size">
+ <term><varname>max_slot_wal_keep_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_slot_wal_keep_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specify the maximum size of WAL files
+ that <link linkend="streaming-replication-slots">replication
+ slots</link> are allowed to retain in the <filename>pg_wal</filename>
+ directory at checkpoint time.
+ If <varname>max_slot_wal_keep_size</varname> is zero (the default),
+ replication slots retain unlimited size of WAL files.
+ </para>
+ <para>
+ This parameter is used being rounded down to the multiples of WAL file
+ size.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-sender-timeout" xreflabel="wal_sender_timeout">
<term><varname>wal_sender_timeout</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index 8cb77f85ec..04cdccb10d 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -927,9 +927,11 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<xref linkend="guc-archive-command"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
- known to be needed. An advantage of these methods is that they bound
- the space requirement for <literal>pg_wal</literal>; there is currently no way
- to do this using replication slots.
+ known to be needed. On the other hand, replication slots can retain so
+ many WAL segments that they fill up the space allotted
+ for <literal>pg_wal</literal>;
+ <xref linkend="guc-max-slot-wal-keep-size"/> limits the size of WAL files
+ retained by replication slots.
</para>
<para>
Similarly, <xref linkend="guc-hot-standby-feedback"/>
--
2.16.3
----Next_Part(Tue_Sep_04_19_52_50_2018_992)----
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH 5/6] Documentation for slot-limit feature
@ 2018-01-11 06:00 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Kyotaro Horiguchi @ 2018-01-11 06:00 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 28 ++++++++++++++++++++++++++++
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
doc/src/sgml/high-availability.sgml | 8 +++++---
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index af4d0625ea..7ec8764ce5 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9825,6 +9825,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</entry>
</row>
+ <row>
+ <entry><structfield>wal_status</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+
+ <entry>Availability of WAL records claimed by this
+ slot. <literal>streaming</literal>, <literal>keeping</literal>,
+ <literal>lost</literal>
+ or <literal>unknown</literal>. <literal>streaming</literal> means that
+ the claimed records are available. <literal>keeping</literal> means that
+ some of them are to be removed by the next checkpoint.
+ <literal>lost</literal> means that some of them are no longer
+ available. The last two states are seen only when
+ <xref linkend="guc-max-slot-wal-keep-size"/> is
+ non-negative. If <structfield>restart_lsn</structfield> is NULL, this
+ field is <literal>unknown</literal>.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>remain</structfield></entry>
+ <entry><type>bigint</type></entry>
+ <entry></entry>
+ <entry>The amount in bytes that WAL location (LSN) can advance until
+ this slot may lose required WAL records.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 4a7121a51f..3d034ac0d1 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3531,6 +3531,29 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-slot-wal-keep-size" xreflabel="max_slot_wal_keep_size">
+ <term><varname>max_slot_wal_keep_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_slot_wal_keep_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specify the maximum size of WAL files
+ that <link linkend="streaming-replication-slots">replication
+ slots</link> are allowed to retain in the <filename>pg_wal</filename>
+ directory at checkpoint time.
+ If <varname>max_slot_wal_keep_size</varname> is -1 (the default),
+ replication slots retain unlimited size of WAL files. If restart_lsn
+ of a replication slot gets behind more than that bytes from the
+ current LSN, the standby using the slot may no longer be able to
+ reconnect due to removal of required WAL records. You can see the WAL
+ availability of replication slots
+ in <link linkend="view-pg-replication-slots">pg_replication_slots</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-sender-timeout" xreflabel="wal_sender_timeout">
<term><varname>wal_sender_timeout</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index d8fd195da0..e30eaaeebe 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -925,9 +925,11 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<xref linkend="guc-archive-command"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
- known to be needed. An advantage of these methods is that they bound
- the space requirement for <literal>pg_wal</literal>; there is currently no way
- to do this using replication slots.
+ known to be needed. On the other hand, replication slots can retain so
+ many WAL segments that they fill up the space allocated
+ for <literal>pg_wal</literal>;
+ <xref linkend="guc-max-slot-wal-keep-size"/> limits the size of WAL files
+ retained by replication slots.
</para>
<para>
Similarly, <xref linkend="guc-hot-standby-feedback"/>
--
2.16.3
----Next_Part(Thu_Dec_20_16_24_38_2018_792)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v11-0006-Check-removal-of-in-reading-segment-file.patch"
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH 4/4] Documentation for slot-limit feature
@ 2018-01-11 06:00 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Kyotaro Horiguchi @ 2018-01-11 06:00 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 28 ++++++++++++++++++++++++++++
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
doc/src/sgml/high-availability.sgml | 8 +++++---
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9edba96fab..fc4cbc9239 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9881,6 +9881,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</entry>
</row>
+ <row>
+ <entry><structfield>wal_status</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+
+ <entry>Availability of WAL records claimed by the
+ slot. <literal>streaming</literal>, <literal>keeping</literal>,
+ <literal>lost</literal>
+ or <literal>unknown</literal>. <literal>streaming</literal> means that
+ the claimed records are available. <literal>keeping</literal> means that
+ some of them are to be removed by the next checkpoint.
+ <literal>lost</literal> means that some of them are no longer
+ available. The last two states are seen only when
+ <xref linkend="guc-max-slot-wal-keep-size"/> is non-negative. If the
+ slot doesn't have valid restart_lsn, this field
+ is <literal>unknown</literal>.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>remain</structfield></entry>
+ <entry><type>bigint</type></entry>
+ <entry></entry>
+ <entry>The amount in bytes that WAL location (LSN) can advance until the
+ slot may lose required WAL records.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 7554cba3f9..f3e504862c 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3117,6 +3117,29 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-slot-wal-keep-size" xreflabel="max_slot_wal_keep_size">
+ <term><varname>max_slot_wal_keep_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_slot_wal_keep_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specify the maximum size of WAL files
+ that <link linkend="streaming-replication-slots">replication
+ slots</link> are allowed to retain in the <filename>pg_wal</filename>
+ directory at checkpoint time.
+ If <varname>max_slot_wal_keep_size</varname> is -1 (the default),
+ replication slots retain unlimited size of WAL files. If restart_lsn
+ of a replication slot gets behind more than that bytes from the
+ current LSN, the standby using the slot may no longer be able to
+ reconnect due to removal of required WAL records. You can see the WAL
+ availability of replication slots
+ in <link linkend="view-pg-replication-slots">pg_replication_slots</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-sender-timeout" xreflabel="wal_sender_timeout">
<term><varname>wal_sender_timeout</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index ebcb3daaed..15a98340a6 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -927,9 +927,11 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<xref linkend="guc-archive-command"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
- known to be needed. An advantage of these methods is that they bound
- the space requirement for <literal>pg_wal</literal>; there is currently no way
- to do this using replication slots.
+ known to be needed. On the other hand, replication slots can retain so
+ many WAL segments that they fill up the space allotted
+ for <literal>pg_wal</literal>;
+ <xref linkend="guc-max-slot-wal-keep-size"/> limits the size of WAL files
+ retained by replication slots.
</para>
<para>
Similarly, <xref linkend="guc-hot-standby-feedback"/>
--
2.16.3
----Next_Part(Thu_Oct_25_21_55_18_2018_318)----
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH 5/6] Documentation for slot-limit feature
@ 2018-01-11 06:00 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Kyotaro Horiguchi @ 2018-01-11 06:00 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 28 ++++++++++++++++++++++++++++
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
doc/src/sgml/high-availability.sgml | 8 +++++---
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index af4d0625ea..7ec8764ce5 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9825,6 +9825,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</entry>
</row>
+ <row>
+ <entry><structfield>wal_status</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+
+ <entry>Availability of WAL records claimed by this
+ slot. <literal>streaming</literal>, <literal>keeping</literal>,
+ <literal>lost</literal>
+ or <literal>unknown</literal>. <literal>streaming</literal> means that
+ the claimed records are available. <literal>keeping</literal> means that
+ some of them are to be removed by the next checkpoint.
+ <literal>lost</literal> means that some of them are no longer
+ available. The last two states are seen only when
+ <xref linkend="guc-max-slot-wal-keep-size"/> is
+ non-negative. If <structfield>restart_lsn</structfield> is NULL, this
+ field is <literal>unknown</literal>.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>remain</structfield></entry>
+ <entry><type>bigint</type></entry>
+ <entry></entry>
+ <entry>The amount in bytes that WAL location (LSN) can advance until
+ this slot may lose required WAL records.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index b6f5822b84..7177c6122a 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3537,6 +3537,29 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-slot-wal-keep-size" xreflabel="max_slot_wal_keep_size">
+ <term><varname>max_slot_wal_keep_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_slot_wal_keep_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specify the maximum size of WAL files
+ that <link linkend="streaming-replication-slots">replication
+ slots</link> are allowed to retain in the <filename>pg_wal</filename>
+ directory at checkpoint time.
+ If <varname>max_slot_wal_keep_size</varname> is -1 (the default),
+ replication slots retain unlimited size of WAL files. If restart_lsn
+ of a replication slot gets behind more than that bytes from the
+ current LSN, the standby using the slot may no longer be able to
+ reconnect due to removal of required WAL records. You can see the WAL
+ availability of replication slots
+ in <link linkend="view-pg-replication-slots">pg_replication_slots</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-sender-timeout" xreflabel="wal_sender_timeout">
<term><varname>wal_sender_timeout</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index bbab7395a2..79901c5f06 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -925,9 +925,11 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<xref linkend="guc-archive-command"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
- known to be needed. An advantage of these methods is that they bound
- the space requirement for <literal>pg_wal</literal>; there is currently no way
- to do this using replication slots.
+ known to be needed. On the other hand, replication slots can retain so
+ many WAL segments that they fill up the space allocated
+ for <literal>pg_wal</literal>;
+ <xref linkend="guc-max-slot-wal-keep-size"/> limits the size of WAL files
+ retained by replication slots.
</para>
<para>
Similarly, <xref linkend="guc-hot-standby-feedback"/>
--
2.16.3
----Next_Part(Wed_Jan_30_10_42_04_2019_681)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v12-0006-Check-removal-of-in-reading-segment-file.patch"
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH 4/4] Documentation for slot-limit feature
@ 2018-01-11 06:00 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Kyotaro Horiguchi @ 2018-01-11 06:00 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 28 ++++++++++++++++++++++++++++
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
doc/src/sgml/high-availability.sgml | 8 +++++---
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0179deea2e..84a937e1fe 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9879,6 +9879,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</entry>
</row>
+ <row>
+ <entry><structfield>wal_status</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+
+ <entry>Availability of WAL records claimed by the
+ slot. <literal>streaming</literal>, <literal>keeping</literal>,
+ <literal>lost</literal>
+ or <literal>unknown</literal>. <literal>streaming</literal> means that
+ the claimed records are available. <literal>keeping</literal> means that
+ some of them are to be removed by the next checkpoint.
+ <literal>lost</literal> means that some of them are no longer
+ available. The last two states are seen only when
+ <xref linkend="guc-max-slot-wal-keep-size"/> is non-negative. If the
+ slot doesn't have valid restart_lsn, this field
+ is <literal>unknown</literal>.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>remain</structfield></entry>
+ <entry><type>bigint</type></entry>
+ <entry></entry>
+ <entry>The amount in bytes that WAL location (LSN) can advance until the
+ slot may lose required WAL records.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index bee4afbe4e..edd5419ec6 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3116,6 +3116,29 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-slot-wal-keep-size" xreflabel="max_slot_wal_keep_size">
+ <term><varname>max_slot_wal_keep_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_slot_wal_keep_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specify the maximum size of WAL files
+ that <link linkend="streaming-replication-slots">replication
+ slots</link> are allowed to retain in the <filename>pg_wal</filename>
+ directory at checkpoint time.
+ If <varname>max_slot_wal_keep_size</varname> is -1 (the default),
+ replication slots retain unlimited size of WAL files. If restart_lsn
+ of a replication slot gets behind more than that bytes from the
+ current LSN, the standby using the slot may no longer be able to
+ reconnect due to removal of required WAL records. You can see the WAL
+ availability of replication slots
+ in <link linkend="view-pg-replication-slots">pg_replication_slots</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-wal-sender-timeout" xreflabel="wal_sender_timeout">
<term><varname>wal_sender_timeout</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index 8cb77f85ec..04cdccb10d 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -927,9 +927,11 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<xref linkend="guc-archive-command"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
- known to be needed. An advantage of these methods is that they bound
- the space requirement for <literal>pg_wal</literal>; there is currently no way
- to do this using replication slots.
+ known to be needed. On the other hand, replication slots can retain so
+ many WAL segments that they fill up the space allotted
+ for <literal>pg_wal</literal>;
+ <xref linkend="guc-max-slot-wal-keep-size"/> limits the size of WAL files
+ retained by replication slots.
</para>
<para>
Similarly, <xref linkend="guc-hot-standby-feedback"/>
--
2.16.3
----Next_Part(Thu_Sep_13_18_29_31_2018_032)----
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH] Align timestamps in pg_regress output
@ 2019-02-21 09:35 Christoph Berg <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Christoph Berg @ 2019-02-21 09:35 UTC (permalink / raw)
---
src/test/regress/pg_regress.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index a18a6f6c45..8111d95b1e 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -1799,7 +1799,7 @@ run_schedule(const char *schedule, test_function tfunc)
}
else
{
- status(_("ok"));
+ status(_("ok ")); /* align with FAILED */
success_count++;
}
@@ -1879,7 +1879,7 @@ run_single_test(const char *test, test_function tfunc)
}
else
{
- status(_("ok"));
+ status(_("ok ")); /* align with FAILED */
success_count++;
}
--
2.20.1
--r5Pyd7+fXNt84Ff3--
^ permalink raw reply [nested|flat] 72+ messages in thread
* [PATCH v22 7/8] Row pattern recognition patch (tests).
@ 2024-09-19 04:48 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Tatsuo Ishii @ 2024-09-19 04:48 UTC (permalink / raw)
---
src/test/regress/expected/rpr.out | 836 +++++++++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/rpr.sql | 405 ++++++++++++++
3 files changed, 1242 insertions(+), 1 deletion(-)
create mode 100644 src/test/regress/expected/rpr.out
create mode 100644 src/test/regress/sql/rpr.sql
diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out
new file mode 100644
index 0000000000..0789e09435
--- /dev/null
+++ b/src/test/regress/expected/rpr.out
@@ -0,0 +1,836 @@
+--
+-- Test for row pattern definition clause
+--
+CREATE TEMP TABLE stock (
+ company TEXT,
+ tdate DATE,
+ price INTEGER
+);
+INSERT INTO stock VALUES ('company1', '2023-07-01', 100);
+INSERT INTO stock VALUES ('company1', '2023-07-02', 200);
+INSERT INTO stock VALUES ('company1', '2023-07-03', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-04', 140);
+INSERT INTO stock VALUES ('company1', '2023-07-05', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-06', 90);
+INSERT INTO stock VALUES ('company1', '2023-07-07', 110);
+INSERT INTO stock VALUES ('company1', '2023-07-08', 130);
+INSERT INTO stock VALUES ('company1', '2023-07-09', 120);
+INSERT INTO stock VALUES ('company1', '2023-07-10', 130);
+INSERT INTO stock VALUES ('company2', '2023-07-01', 50);
+INSERT INTO stock VALUES ('company2', '2023-07-02', 2000);
+INSERT INTO stock VALUES ('company2', '2023-07-03', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-04', 1400);
+INSERT INTO stock VALUES ('company2', '2023-07-05', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-06', 60);
+INSERT INTO stock VALUES ('company2', '2023-07-07', 1100);
+INSERT INTO stock VALUES ('company2', '2023-07-08', 1300);
+INSERT INTO stock VALUES ('company2', '2023-07-09', 1200);
+INSERT INTO stock VALUES ('company2', '2023-07-10', 1300);
+SELECT * FROM stock;
+ company | tdate | price
+----------+------------+-------
+ company1 | 07-01-2023 | 100
+ company1 | 07-02-2023 | 200
+ company1 | 07-03-2023 | 150
+ company1 | 07-04-2023 | 140
+ company1 | 07-05-2023 | 150
+ company1 | 07-06-2023 | 90
+ company1 | 07-07-2023 | 110
+ company1 | 07-08-2023 | 130
+ company1 | 07-09-2023 | 120
+ company1 | 07-10-2023 | 130
+ company2 | 07-01-2023 | 50
+ company2 | 07-02-2023 | 2000
+ company2 | 07-03-2023 | 1500
+ company2 | 07-04-2023 | 1400
+ company2 | 07-05-2023 | 1500
+ company2 | 07-06-2023 | 60
+ company2 | 07-07-2023 | 1100
+ company2 | 07-08-2023 | 1300
+ company2 | 07-09-2023 | 1200
+ company2 | 07-10-2023 | 1300
+(20 rows)
+
+-- basic test using PREV
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | | |
+ company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023
+ company1 | 07-07-2023 | 110 | | |
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | | |
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023
+ company2 | 07-07-2023 | 1100 | | |
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- basic test using PREV. UP appears twice
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+ UP+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 150 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | | |
+ company1 | 07-06-2023 | 90 | 90 | 130 | 07-07-2023
+ company1 | 07-07-2023 | 110 | | |
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1500 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | | |
+ company2 | 07-06-2023 | 60 | 60 | 1300 | 07-07-2023
+ company2 | 07-07-2023 | 1100 | | |
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- basic test using PREV. Use '*'
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP* DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | 150 | 90 | 07-06-2023
+ company1 | 07-06-2023 | 90 | | |
+ company1 | 07-07-2023 | 110 | 110 | 120 | 07-08-2023
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | 1500 | 60 | 07-06-2023
+ company2 | 07-06-2023 | 60 | | |
+ company2 | 07-07-2023 | 1100 | 1100 | 1200 | 07-08-2023
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- basic test with none greedy pattern
+SELECT company, tdate, price, count(*) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (A A A)
+ DEFINE
+ A AS price >= 140 AND price <= 150
+);
+ company | tdate | price | count
+----------+------------+-------+-------
+ company1 | 07-01-2023 | 100 | 0
+ company1 | 07-02-2023 | 200 | 0
+ company1 | 07-03-2023 | 150 | 3
+ company1 | 07-04-2023 | 140 | 0
+ company1 | 07-05-2023 | 150 | 0
+ company1 | 07-06-2023 | 90 | 0
+ company1 | 07-07-2023 | 110 | 0
+ company1 | 07-08-2023 | 130 | 0
+ company1 | 07-09-2023 | 120 | 0
+ company1 | 07-10-2023 | 130 | 0
+ company2 | 07-01-2023 | 50 | 0
+ company2 | 07-02-2023 | 2000 | 0
+ company2 | 07-03-2023 | 1500 | 0
+ company2 | 07-04-2023 | 1400 | 0
+ company2 | 07-05-2023 | 1500 | 0
+ company2 | 07-06-2023 | 60 | 0
+ company2 | 07-07-2023 | 1100 | 0
+ company2 | 07-08-2023 | 1300 | 0
+ company2 | 07-09-2023 | 1200 | 0
+ company2 | 07-10-2023 | 1300 | 0
+(20 rows)
+
+-- last_value() should remain consistent
+SELECT company, tdate, price, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | last_value
+----------+------------+-------+------------
+ company1 | 07-01-2023 | 100 | 140
+ company1 | 07-02-2023 | 200 |
+ company1 | 07-03-2023 | 150 |
+ company1 | 07-04-2023 | 140 |
+ company1 | 07-05-2023 | 150 |
+ company1 | 07-06-2023 | 90 | 120
+ company1 | 07-07-2023 | 110 |
+ company1 | 07-08-2023 | 130 |
+ company1 | 07-09-2023 | 120 |
+ company1 | 07-10-2023 | 130 |
+ company2 | 07-01-2023 | 50 | 1400
+ company2 | 07-02-2023 | 2000 |
+ company2 | 07-03-2023 | 1500 |
+ company2 | 07-04-2023 | 1400 |
+ company2 | 07-05-2023 | 1500 |
+ company2 | 07-06-2023 | 60 | 1200
+ company2 | 07-07-2023 | 1100 |
+ company2 | 07-08-2023 | 1300 |
+ company2 | 07-09-2023 | 1200 |
+ company2 | 07-10-2023 | 1300 |
+(20 rows)
+
+-- omit "START" in DEFINE but it is ok because "START AS TRUE" is
+-- implicitly defined. per spec.
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | | |
+ company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023
+ company1 | 07-07-2023 | 110 | | |
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | | |
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023
+ company2 | 07-07-2023 | 1100 | | |
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- the first row start with less than or equal to 100
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | 90 | 120
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1400
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | 60 | 1200
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- second row raises 120%
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price) * 1.2,
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1400
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- using NEXT
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 200
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | 140 | 150
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 110 | 130
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 2000
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | 1400 | 1500
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 1100 | 1300
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 200
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | 140 | 150
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 110 | 130
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 2000
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | 1400 | 1500
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 1100 | 1300
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- match everything
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+)
+ DEFINE
+ A AS TRUE
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 130
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1300
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | |
+ company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | |
+ company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | |
+ company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023
+ company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023
+ company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023
+ company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023
+ company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | |
+ company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023
+ company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023
+ company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023
+ company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023
+ company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w,
+ count(*) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | count
+----------+------------+-------+-------------+------------+-------
+ company1 | 07-01-2023 | 100 | 07-01-2023 | 07-03-2023 | 3
+ company1 | 07-02-2023 | 200 | | | 0
+ company1 | 07-03-2023 | 150 | | | 0
+ company1 | 07-04-2023 | 140 | 07-04-2023 | 07-06-2023 | 3
+ company1 | 07-05-2023 | 150 | | | 0
+ company1 | 07-06-2023 | 90 | | | 0
+ company1 | 07-07-2023 | 110 | 07-07-2023 | 07-09-2023 | 3
+ company1 | 07-08-2023 | 130 | | | 0
+ company1 | 07-09-2023 | 120 | | | 0
+ company1 | 07-10-2023 | 130 | | | 0
+ company2 | 07-01-2023 | 50 | 07-01-2023 | 07-03-2023 | 3
+ company2 | 07-02-2023 | 2000 | | | 0
+ company2 | 07-03-2023 | 1500 | | | 0
+ company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-06-2023 | 3
+ company2 | 07-05-2023 | 1500 | | | 0
+ company2 | 07-06-2023 | 60 | | | 0
+ company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-09-2023 | 3
+ company2 | 07-08-2023 | 1300 | | | 0
+ company2 | 07-09-2023 | 1200 | | | 0
+ company2 | 07-10-2023 | 1300 | | | 0
+(20 rows)
+
+--
+-- Aggregates
+--
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP PAST LAST ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | max | min | sum | avg | count
+----------+------------+-------+-------------+------------+------+-----+------+-----------------------+-------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4
+ company1 | 07-02-2023 | 200 | | | | | | | 0
+ company1 | 07-03-2023 | 150 | | | | | | | 0
+ company1 | 07-04-2023 | 140 | | | | | | | 0
+ company1 | 07-05-2023 | 150 | | | | | | | 0
+ company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4
+ company1 | 07-07-2023 | 110 | | | | | | | 0
+ company1 | 07-08-2023 | 130 | | | | | | | 0
+ company1 | 07-09-2023 | 120 | | | | | | | 0
+ company1 | 07-10-2023 | 130 | | | | | | | 0
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4
+ company2 | 07-02-2023 | 2000 | | | | | | | 0
+ company2 | 07-03-2023 | 1500 | | | | | | | 0
+ company2 | 07-04-2023 | 1400 | | | | | | | 0
+ company2 | 07-05-2023 | 1500 | | | | | | | 0
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4
+ company2 | 07-07-2023 | 1100 | | | | | | | 0
+ company2 | 07-08-2023 | 1300 | | | | | | | 0
+ company2 | 07-09-2023 | 1200 | | | | | | | 0
+ company2 | 07-10-2023 | 1300 | | | | | | | 0
+(20 rows)
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP TO NEXT ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | max | min | sum | avg | count
+----------+------------+-------+-------------+------------+------+------+------+-----------------------+-------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4
+ company1 | 07-02-2023 | 200 | | | | | | | 0
+ company1 | 07-03-2023 | 150 | | | | | | | 0
+ company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3
+ company1 | 07-05-2023 | 150 | | | | | | | 0
+ company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4
+ company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3
+ company1 | 07-08-2023 | 130 | | | | | | | 0
+ company1 | 07-09-2023 | 120 | | | | | | | 0
+ company1 | 07-10-2023 | 130 | | | | | | | 0
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4
+ company2 | 07-02-2023 | 2000 | | | | | | | 0
+ company2 | 07-03-2023 | 1500 | | | | | | | 0
+ company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3
+ company2 | 07-05-2023 | 1500 | | | | | | | 0
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4
+ company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3
+ company2 | 07-08-2023 | 1300 | | | | | | | 0
+ company2 | 07-09-2023 | 1200 | | | | | | | 0
+ company2 | 07-10-2023 | 1300 | | | | | | | 0
+(20 rows)
+
+-- JOIN case
+CREATE TEMP TABLE t1 (i int, v1 int);
+CREATE TEMP TABLE t2 (j int, v2 int);
+INSERT INTO t1 VALUES(1,10);
+INSERT INTO t1 VALUES(1,11);
+INSERT INTO t1 VALUES(1,12);
+INSERT INTO t2 VALUES(2,10);
+INSERT INTO t2 VALUES(2,11);
+INSERT INTO t2 VALUES(2,12);
+SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11;
+ i | v1 | j | v2
+---+----+---+----
+ 1 | 10 | 2 | 10
+ 1 | 10 | 2 | 11
+ 1 | 11 | 2 | 10
+ 1 | 11 | 2 | 11
+(4 rows)
+
+SELECT *, count(*) OVER w FROM t1, t2
+WINDOW w AS (
+ PARTITION BY t1.i
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (A)
+ DEFINE
+ A AS v1 <= 11 AND v2 <= 11
+);
+ i | v1 | j | v2 | count
+---+----+---+----+-------
+ 1 | 10 | 2 | 10 | 1
+ 1 | 10 | 2 | 11 | 1
+ 1 | 10 | 2 | 12 | 0
+ 1 | 11 | 2 | 10 | 1
+ 1 | 11 | 2 | 11 | 1
+ 1 | 11 | 2 | 12 | 0
+ 1 | 12 | 2 | 10 | 0
+ 1 | 12 | 2 | 11 | 0
+ 1 | 12 | 2 | 12 | 0
+(9 rows)
+
+-- WITH case
+WITH wstock AS (
+ SELECT * FROM stock WHERE tdate < '2023-07-08'
+)
+SELECT tdate, price,
+first_value(tdate) OVER w,
+count(*) OVER w
+ FROM wstock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ tdate | price | first_value | count
+------------+-------+-------------+-------
+ 07-01-2023 | 100 | 07-01-2023 | 4
+ 07-02-2023 | 200 | | 0
+ 07-03-2023 | 150 | | 0
+ 07-04-2023 | 140 | | 0
+ 07-05-2023 | 150 | | 0
+ 07-06-2023 | 90 | | 0
+ 07-07-2023 | 110 | | 0
+ 07-01-2023 | 50 | 07-01-2023 | 4
+ 07-02-2023 | 2000 | | 0
+ 07-03-2023 | 1500 | | 0
+ 07-04-2023 | 1400 | | 0
+ 07-05-2023 | 1500 | | 0
+ 07-06-2023 | 60 | | 0
+ 07-07-2023 | 1100 | | 0
+(14 rows)
+
+--
+-- Error cases
+--
+-- row pattern definition variable name must not appear more than once
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price),
+ UP AS price > PREV(price)
+);
+ERROR: row pattern definition variable name "up" appears more than once in DEFINE clause
+LINE 11: UP AS price > PREV(price),
+ ^
+-- subqueries in DEFINE clause are not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START LOWPRICE)
+ DEFINE
+ START AS TRUE,
+ LOWPRICE AS price < (SELECT 100)
+);
+ERROR: cannot use subquery in DEFINE expression
+LINE 11: LOWPRICE AS price < (SELECT 100)
+ ^
+-- aggregates in DEFINE clause are not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START LOWPRICE)
+ DEFINE
+ START AS TRUE,
+ LOWPRICE AS price < count(*)
+);
+ERROR: aggregate functions are not allowed in DEFINE
+LINE 11: LOWPRICE AS price < count(*)
+ ^
+-- FRAME must start at current row when row patttern recognition is used
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ERROR: FRAME must start at current row when row patttern recognition is used
+-- SEEK is not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ SEEK
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ERROR: SEEK is not supported
+LINE 8: SEEK
+ ^
+HINT: Use INITIAL.
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4f38104ba0..64604d3003 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -98,7 +98,7 @@ test: publication subscription
# Another group of parallel tests
# select_views depends on create_view
# ----------
-test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass
+test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr
# ----------
# Another group of parallel tests (JSON related)
diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql
new file mode 100644
index 0000000000..302e2b86a5
--- /dev/null
+++ b/src/test/regress/sql/rpr.sql
@@ -0,0 +1,405 @@
+--
+-- Test for row pattern definition clause
+--
+
+CREATE TEMP TABLE stock (
+ company TEXT,
+ tdate DATE,
+ price INTEGER
+);
+INSERT INTO stock VALUES ('company1', '2023-07-01', 100);
+INSERT INTO stock VALUES ('company1', '2023-07-02', 200);
+INSERT INTO stock VALUES ('company1', '2023-07-03', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-04', 140);
+INSERT INTO stock VALUES ('company1', '2023-07-05', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-06', 90);
+INSERT INTO stock VALUES ('company1', '2023-07-07', 110);
+INSERT INTO stock VALUES ('company1', '2023-07-08', 130);
+INSERT INTO stock VALUES ('company1', '2023-07-09', 120);
+INSERT INTO stock VALUES ('company1', '2023-07-10', 130);
+INSERT INTO stock VALUES ('company2', '2023-07-01', 50);
+INSERT INTO stock VALUES ('company2', '2023-07-02', 2000);
+INSERT INTO stock VALUES ('company2', '2023-07-03', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-04', 1400);
+INSERT INTO stock VALUES ('company2', '2023-07-05', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-06', 60);
+INSERT INTO stock VALUES ('company2', '2023-07-07', 1100);
+INSERT INTO stock VALUES ('company2', '2023-07-08', 1300);
+INSERT INTO stock VALUES ('company2', '2023-07-09', 1200);
+INSERT INTO stock VALUES ('company2', '2023-07-10', 1300);
+
+SELECT * FROM stock;
+
+-- basic test using PREV
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- basic test using PREV. UP appears twice
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+ UP+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- basic test using PREV. Use '*'
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP* DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- basic test with none greedy pattern
+SELECT company, tdate, price, count(*) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (A A A)
+ DEFINE
+ A AS price >= 140 AND price <= 150
+);
+
+-- last_value() should remain consistent
+SELECT company, tdate, price, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- omit "START" in DEFINE but it is ok because "START AS TRUE" is
+-- implicitly defined. per spec.
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- the first row start with less than or equal to 100
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- second row raises 120%
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price) * 1.2,
+ DOWN AS price < PREV(price)
+);
+
+-- using NEXT
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+
+-- match everything
+
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+)
+ DEFINE
+ A AS TRUE
+);
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+
+-- ROWS BETWEEN CURRENT ROW AND offset FOLLOWING
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w,
+ count(*) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+--
+-- Aggregates
+--
+
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP PAST LAST ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP TO NEXT ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+
+-- JOIN case
+CREATE TEMP TABLE t1 (i int, v1 int);
+CREATE TEMP TABLE t2 (j int, v2 int);
+INSERT INTO t1 VALUES(1,10);
+INSERT INTO t1 VALUES(1,11);
+INSERT INTO t1 VALUES(1,12);
+INSERT INTO t2 VALUES(2,10);
+INSERT INTO t2 VALUES(2,11);
+INSERT INTO t2 VALUES(2,12);
+
+SELECT * FROM t1, t2 WHERE t1.v1 <= 11 AND t2.v2 <= 11;
+
+SELECT *, count(*) OVER w FROM t1, t2
+WINDOW w AS (
+ PARTITION BY t1.i
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (A)
+ DEFINE
+ A AS v1 <= 11 AND v2 <= 11
+);
+
+-- WITH case
+WITH wstock AS (
+ SELECT * FROM stock WHERE tdate < '2023-07-08'
+)
+SELECT tdate, price,
+first_value(tdate) OVER w,
+count(*) OVER w
+ FROM wstock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+--
+-- Error cases
+--
+
+-- row pattern definition variable name must not appear more than once
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price),
+ UP AS price > PREV(price)
+);
+
+-- subqueries in DEFINE clause are not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START LOWPRICE)
+ DEFINE
+ START AS TRUE,
+ LOWPRICE AS price < (SELECT 100)
+);
+
+-- aggregates in DEFINE clause are not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START LOWPRICE)
+ DEFINE
+ START AS TRUE,
+ LOWPRICE AS price < count(*)
+);
+
+-- FRAME must start at current row when row patttern recognition is used
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- SEEK is not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ SEEK
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
--
2.25.1
----Next_Part(Thu_Sep_19_13_59_47_2024_608)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v22-0008-Allow-to-print-raw-parse-tree.patch"
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
@ 2024-09-20 11:45 Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-23 23:46 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-24 01:37 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-26 06:14 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 4 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-09-20 11:45 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Sep 11, 2024 at 8:55 AM Peter Smith <[email protected]> wrote:
>
> Here are a some more review comments for patch v30-0001.
>
> ======
> src/sgml/ref/create_publication.sgml
>
> 1.
> + <para>
> + If the publisher-side column is also a generated column
> then this option
> + has no effect; the publisher column will be filled as normal with the
> + publisher-side computed or default data.
> + </para>
>
> It should say "subscriber-side"; not "publisher-side". The same was
> already reported by Sawada-San [1].
>
> ~~~
>
> 2.
> + <para>
> + This parameter can only be set <literal>true</literal> if
> <literal>copy_data</literal> is
> + set to <literal>false</literal>.
> + </para>
>
> IMO this limitation should be addressed by patch 0001 like it was
> already done in the previous patches (e.g. v22-0002). I think
> Sawada-san suggested the same [1].
>
> Anyway, 'copy_data' is not a PUBLICATION option, so the fact it is
> mentioned like this without any reference to the SUBSCRIPTION seems
> like a cut/paste error from the previous implementation.
>
> ======
> src/backend/catalog/pg_publication.c
>
> 3. pub_collist_validate
> - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> - ereport(ERROR,
> - errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> - errmsg("cannot use generated column \"%s\" in publication column list",
> - colname));
> -
>
> Instead of just removing this ERROR entirely here, I thought it would
> be more user-friendly to give a WARNING if the PUBLICATION's explicit
> column list includes generated cols when the option
> "publish_generated_columns" is false. This combination doesn't seem
> like something a user would do intentionally, so just silently
> ignoring it (like the current patch does) is likely going to give
> someone unexpected results/grief.
>
> ======
> src/backend/replication/logical/proto.c
>
> 4. logicalrep_write_tuple, and logicalrep_write_attrs:
>
> - if (att->attisdropped || att->attgenerated)
> + if (att->attisdropped)
> continue;
>
> Why aren't you also checking the new PUBLICATION option here and
> skipping all gencols if the "publish_generated_columns" option is
> false? Or is the BMS of pgoutput_column_list_init handling this case?
> Maybe there should be an Assert for this?
>
> ======
> src/backend/replication/pgoutput/pgoutput.c
>
> 5. send_relation_and_attrs
>
> - if (att->attisdropped || att->attgenerated)
> + if (att->attisdropped)
> continue;
>
> Same question as #4.
>
> ~~~
>
> 6. prepare_all_columns_bms and pgoutput_column_list_init
>
> + if (att->attgenerated && !pub->pubgencolumns)
> + cols = bms_del_member(cols, i + 1);
>
> IIUC, the algorithm seems overly tricky filling the BMS with all
> columns, before straight away conditionally removing the generated
> columns. Can't it be refactored to assign all the correct columns
> up-front, to avoid calling bms_del_member()?
>
> ======
> src/bin/pg_dump/pg_dump.c
>
> 7. getPublications
>
> IIUC, there is lots of missing SQL code here (for all older versions)
> that should be saying "false AS pubgencolumns".
> e.g. compare the SQL with how "false AS pubviaroot" is used.
>
> ======
> src/bin/pg_dump/t/002_pg_dump.pl
>
> 8. Missing tests?
>
> I expected to see a pg_dump test for this new PUBLICATION option.
>
> ======
> src/test/regress/sql/publication.sql
>
> 9. Missing tests?
>
> How about adding another test case that checks this new option must be
> "Boolean"?
>
> ~~~
>
> 10. Missing tests?
>
> --- error: generated column "d" can't be in list
> +-- ok: generated columns can be in the list too
> ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
> +ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
>
> (see my earlier comment #3)
>
> IMO there should be another test case for a WARNING here if the user
> attempts to include generated column 'd' in an explicit PUBLICATION
> column list while the "publish_generated-columns" is false.
>
> ======
> [1] https://www.postgresql.org/message-id/CAD21AoA-tdTz0G-vri8KM2TXeFU8RCDsOpBXUBCgwkfokF7%3DjA%40mail.g...
>
I have fixed all the comments. The attached patches contain the desired changes.
Also the merging of 0001 and 0002 can be done once there are no
comments on the patch to help in reviewing.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v32-0002-Support-replication-of-generated-column-during-i.patch (18.4K, ../../CAHv8RjKkoaS1oMsFvPRFB9nPSVC5p_D4Kgq5XB9Y2B2xU7smbA@mail.gmail.com/2-v32-0002-Support-replication-of-generated-column-during-i.patch)
download | inline diff:
From c0d36fa7ca885c9c8bf2813f3d72ebf6383869ed Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Fri, 13 Sep 2024 00:37:06 +0530
Subject: [PATCH v32 2/2] Support replication of generated column during
initial sync
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
when (publish_generated_columns = false)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column:
Publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* publisher generated column => subscriber generated column: Publisher
generated column is not replicated. The subscriber generated column
will be filed with the subscriber-side computed or default data.
---
doc/src/sgml/ref/create_publication.sgml | 4 -
src/backend/catalog/pg_subscription.c | 31 +++
src/backend/commands/subscriptioncmds.c | 31 ---
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 203 ++++++++++++++++----
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
7 files changed, 205 insertions(+), 73 deletions(-)
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index e133dc30d7..1973857586 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -235,10 +235,6 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
This option is only available for replicating generated column data from the publisher
to a regular, non-generated column in the subscriber.
</para>
- <para>
- This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
- set to <literal>false</literal>.
- </para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc636b8..addf307cb6 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -439,37 +439,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
-/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
/*
* Check that the specified publications are present on the publisher.
*/
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..0e34d7cd66 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,25 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ StringInfo pub_names = makeStringInfo();
+
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +931,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +987,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1035,33 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ if (server_version >= 120000)
+ {
+ has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+ }
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1073,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -1005,6 +1106,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1015,7 +1119,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1141,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1227,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1242,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1298,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1369,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..158b444275 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
--
2.41.0.windows.3
[application/octet-stream] v32-0001-Enable-support-for-publish_generated_columns-opt.patch (82.3K, ../../CAHv8RjKkoaS1oMsFvPRFB9nPSVC5p_D4Kgq5XB9Y2B2xU7smbA@mail.gmail.com/3-v32-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 5bafc346dc8e85fa3c6f6c8c93deeb095c3eb042 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Fri, 23 Aug 2024 10:03:38 +1000
Subject: [PATCH v32 1/2] Enable support for 'publish_generated_columns'
option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit enables support for the 'publish_generated_columns' option in
logical replication, allowing the transmission of generated column information
and data alongside regular table changes.
The option 'publish_generated_columns' is a PUBLICATION parameter.
When 'publish_generated_columns' is false, generated columns are not
replicated, even when present in a PUBLICATION col-list.
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 5 -
doc/src/sgml/ref/create_publication.sgml | 20 +
src/backend/catalog/pg_publication.c | 15 +-
src/backend/commands/publicationcmds.c | 36 +-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/pgoutput/pgoutput.c | 93 ++--
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 36 ++
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/pg_publication.h | 7 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 452 +++++++++++---------
src/test/regress/sql/publication.sql | 17 +-
src/test/subscription/t/031_column_list.pl | 36 +-
17 files changed, 475 insertions(+), 298 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index b671858627..2e7804ef24 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 11b6456779..12ffcfb893 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6541,11 +6541,6 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</varlistentry>
</variablelist>
- <para>
- Next, the following message part appears for each column included in
- the publication (except generated columns):
- </para>
-
<variablelist>
<varlistentry>
<term>Int8</term>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..e133dc30d7 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,26 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-include-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ This option is only available for replicating generated column data from the publisher
+ to a regular, non-generated column in the subscriber.
+ </para>
+ <para>
+ This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
+ set to <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2b86..cc12ef36e7 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -420,7 +420,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pub->pubgencols);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -507,7 +508,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
{
Bitmapset *set = NULL;
ListCell *lc;
@@ -531,9 +532,9 @@ pub_collist_validate(Relation targetrel, List *columns)
colname));
if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list for publication with publish_generated_columns as false",
colname));
if (bms_is_member(attnum, set))
@@ -1006,6 +1007,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1216,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..8c09125170 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
@@ -1182,7 +1209,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..1f47ee78e8 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,35 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Prepare new column list bitmap. This includes all the columns of the table.
+ */
+static Bitmapset *
+prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,11 +1071,11 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * If the publication is FOR ALL TABLES and include generated columns
+ * then it is treated the same as if there are no column lists (even
+ * if other publications have a list).
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
{
bool pub_no_list = true;
@@ -1067,43 +1096,47 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
+ }
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ /* Build the column list bitmap in the per-entry context. */
+ if (!pub_no_list || !pub->pubgencols) /* when not null */
+ {
+ int i;
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- pgoutput_ensure_entry_cxt(data, entry);
+ pgoutput_ensure_entry_cxt(data, entry);
+ if (!pub_no_list)
cols = pub_collist_to_bitmapset(cols, cfdatum,
entry->entry_cxt);
+ else
+ cols = prepare_all_columns_bms(data, entry, desc);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ /* Get the number of live attributes. */
+ for (i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
+ if (att->attisdropped)
+ continue;
- nliveatts++;
- }
+ nliveatts++;
+ }
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
}
+ }
+ if (HeapTupleIsValid(cftuple))
ReleaseSysCache(cftuple);
- }
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 130b80775d..d1f0f36c38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4293,7 +4294,13 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
@@ -4326,6 +4333,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4350,6 +4358,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4429,6 +4439,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..2eb16f1f18 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
@@ -3127,6 +3137,32 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'ALTER PUBLICATION pub5 ADD TABLE test_table WHERE (col1 > 0);' => {
+ create_order => 51,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_table WHERE (col1 > 0);',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_table WHERE ((col1 > 0));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => {
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ },
+ },
+
+ 'ALTER PUBLICATION pub5 ADD TABLE test_second_table WHERE (col2 = \'test\');'
+ => {
+ create_order => 52,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_second_table WHERE (col2 = \'test\');',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_second_table WHERE ((col2 = 'test'::text));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => { exclude_dump_test_schema => 1, },
+ },
+
'CREATE SCHEMA public' => {
regexp => qr/^CREATE SCHEMA public;/m,
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index faabecbc76..bfc978de7e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a7ccde6d7d..ea36b18ea2 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3182,7 +3182,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..2a3816f661 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -152,7 +156,8 @@ extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool pubgencols);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..f060cefe2b 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,10 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+WARNING: specified generated column "d" in publication column list for publication with publish_generated_columns as false
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +737,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +924,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1132,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1173,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1254,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1267,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1296,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1322,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1393,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1404,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1425,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1437,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1449,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1460,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1471,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1482,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1513,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1525,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1607,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1628,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1756,27 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..51b6d46940 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1110,6 +1113,18 @@ DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..68c7b2962d 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,17 +1202,17 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int GENERATED ALWAYS AS (a + 1) STORED);
ALTER TABLE test_mix_4 DROP COLUMN c;
- CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 (a, b);
- CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4;
+ CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 WITH (publish_generated_columns = true);
+ CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4 WITH (publish_generated_columns = false);
-- initial data
INSERT INTO test_mix_4 VALUES (1, 2);
@@ -1224,31 +1224,14 @@ $node_subscriber->safe_psql(
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int);
));
-$node_subscriber->safe_psql(
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql(
'postgres', qq(
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub_mix_7, pub_mix_8;
));
-$node_subscriber->wait_for_subscription_sync;
-
-is( $node_subscriber->safe_psql(
- 'postgres', "SELECT * FROM test_mix_4 ORDER BY a"),
- qq(1|2||),
- 'initial synchronization with multiple publications with the same column list'
-);
-
-$node_publisher->safe_psql(
- 'postgres', qq(
- INSERT INTO test_mix_4 VALUES (3, 4);
-));
-
-$node_publisher->wait_for_catchup('sub1');
-
-is( $node_subscriber->safe_psql(
- 'postgres', "SELECT * FROM test_mix_4 ORDER BY a"),
- qq(1|2||
-3|4||),
- 'replication with multiple publications with the same column list');
+ok( $stderr =~
+ qr/cannot use different column lists for table "public.test_mix_4" in different publications/,
+ 'different column lists detected');
# TEST: With a table included in multiple publications with different column
# lists, we should catch the error when creating the subscription.
@@ -1262,11 +1245,10 @@ $node_publisher->safe_psql(
$node_subscriber->safe_psql(
'postgres', qq(
- DROP SUBSCRIPTION sub1;
CREATE TABLE test_mix_1 (a int PRIMARY KEY, b int, c int);
));
-my ($cmdret, $stdout, $stderr) = $node_subscriber->psql(
+($cmdret, $stdout, $stderr) = $node_subscriber->psql(
'postgres', qq(
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub_mix_1, pub_mix_2;
));
--
2.41.0.windows.3
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-23 12:16 ` vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
3 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-09-23 12:16 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, 20 Sept 2024 at 17:15, Shubham Khanna
<[email protected]> wrote:
>
> On Wed, Sep 11, 2024 at 8:55 AM Peter Smith <[email protected]> wrote:
> >
> > Here are a some more review comments for patch v30-0001.
> >
> > ======
> > src/sgml/ref/create_publication.sgml
> >
> > 1.
> > + <para>
> > + If the publisher-side column is also a generated column
> > then this option
> > + has no effect; the publisher column will be filled as normal with the
> > + publisher-side computed or default data.
> > + </para>
> >
> > It should say "subscriber-side"; not "publisher-side". The same was
> > already reported by Sawada-San [1].
> >
> > ~~~
> >
> > 2.
> > + <para>
> > + This parameter can only be set <literal>true</literal> if
> > <literal>copy_data</literal> is
> > + set to <literal>false</literal>.
> > + </para>
> >
> > IMO this limitation should be addressed by patch 0001 like it was
> > already done in the previous patches (e.g. v22-0002). I think
> > Sawada-san suggested the same [1].
> >
> > Anyway, 'copy_data' is not a PUBLICATION option, so the fact it is
> > mentioned like this without any reference to the SUBSCRIPTION seems
> > like a cut/paste error from the previous implementation.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 3. pub_collist_validate
> > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > - ereport(ERROR,
> > - errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > - errmsg("cannot use generated column \"%s\" in publication column list",
> > - colname));
> > -
> >
> > Instead of just removing this ERROR entirely here, I thought it would
> > be more user-friendly to give a WARNING if the PUBLICATION's explicit
> > column list includes generated cols when the option
> > "publish_generated_columns" is false. This combination doesn't seem
> > like something a user would do intentionally, so just silently
> > ignoring it (like the current patch does) is likely going to give
> > someone unexpected results/grief.
> >
> > ======
> > src/backend/replication/logical/proto.c
> >
> > 4. logicalrep_write_tuple, and logicalrep_write_attrs:
> >
> > - if (att->attisdropped || att->attgenerated)
> > + if (att->attisdropped)
> > continue;
> >
> > Why aren't you also checking the new PUBLICATION option here and
> > skipping all gencols if the "publish_generated_columns" option is
> > false? Or is the BMS of pgoutput_column_list_init handling this case?
> > Maybe there should be an Assert for this?
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 5. send_relation_and_attrs
> >
> > - if (att->attisdropped || att->attgenerated)
> > + if (att->attisdropped)
> > continue;
> >
> > Same question as #4.
> >
> > ~~~
> >
> > 6. prepare_all_columns_bms and pgoutput_column_list_init
> >
> > + if (att->attgenerated && !pub->pubgencolumns)
> > + cols = bms_del_member(cols, i + 1);
> >
> > IIUC, the algorithm seems overly tricky filling the BMS with all
> > columns, before straight away conditionally removing the generated
> > columns. Can't it be refactored to assign all the correct columns
> > up-front, to avoid calling bms_del_member()?
> >
> > ======
> > src/bin/pg_dump/pg_dump.c
> >
> > 7. getPublications
> >
> > IIUC, there is lots of missing SQL code here (for all older versions)
> > that should be saying "false AS pubgencolumns".
> > e.g. compare the SQL with how "false AS pubviaroot" is used.
> >
> > ======
> > src/bin/pg_dump/t/002_pg_dump.pl
> >
> > 8. Missing tests?
> >
> > I expected to see a pg_dump test for this new PUBLICATION option.
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 9. Missing tests?
> >
> > How about adding another test case that checks this new option must be
> > "Boolean"?
> >
> > ~~~
> >
> > 10. Missing tests?
> >
> > --- error: generated column "d" can't be in list
> > +-- ok: generated columns can be in the list too
> > ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
> > +ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
> >
> > (see my earlier comment #3)
> >
> > IMO there should be another test case for a WARNING here if the user
> > attempts to include generated column 'd' in an explicit PUBLICATION
> > column list while the "publish_generated-columns" is false.
> >
> > ======
> > [1] https://www.postgresql.org/message-id/CAD21AoA-tdTz0G-vri8KM2TXeFU8RCDsOpBXUBCgwkfokF7%3DjA%40mail.g...
> >
>
> I have fixed all the comments. The attached patches contain the desired changes.
> Also the merging of 0001 and 0002 can be done once there are no
> comments on the patch to help in reviewing.
The warning message appears to be incorrect. Even though
publish_generated_columns is set to true, the warning indicates that
it is false.
CREATE TABLE t1 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
postgres=# CREATE PUBLICATION pub1 FOR table t1(gen1) WITH
(publish_generated_columns=true);
WARNING: specified generated column "gen1" in publication column list
for publication with publish_generated_columns as false
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-09-27 14:31 ` Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-30 07:26 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-30 08:33 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 3 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-09-27 14:31 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Sep 23, 2024 at 5:56 PM vignesh C <[email protected]> wrote:
>
> On Fri, 20 Sept 2024 at 17:15, Shubham Khanna
> <[email protected]> wrote:
> >
> > On Wed, Sep 11, 2024 at 8:55 AM Peter Smith <[email protected]> wrote:
> > >
> > > Here are a some more review comments for patch v30-0001.
> > >
> > > ======
> > > src/sgml/ref/create_publication.sgml
> > >
> > > 1.
> > > + <para>
> > > + If the publisher-side column is also a generated column
> > > then this option
> > > + has no effect; the publisher column will be filled as normal with the
> > > + publisher-side computed or default data.
> > > + </para>
> > >
> > > It should say "subscriber-side"; not "publisher-side". The same was
> > > already reported by Sawada-San [1].
> > >
> > > ~~~
> > >
> > > 2.
> > > + <para>
> > > + This parameter can only be set <literal>true</literal> if
> > > <literal>copy_data</literal> is
> > > + set to <literal>false</literal>.
> > > + </para>
> > >
> > > IMO this limitation should be addressed by patch 0001 like it was
> > > already done in the previous patches (e.g. v22-0002). I think
> > > Sawada-san suggested the same [1].
> > >
> > > Anyway, 'copy_data' is not a PUBLICATION option, so the fact it is
> > > mentioned like this without any reference to the SUBSCRIPTION seems
> > > like a cut/paste error from the previous implementation.
> > >
> > > ======
> > > src/backend/catalog/pg_publication.c
> > >
> > > 3. pub_collist_validate
> > > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > > - ereport(ERROR,
> > > - errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > > - errmsg("cannot use generated column \"%s\" in publication column list",
> > > - colname));
> > > -
> > >
> > > Instead of just removing this ERROR entirely here, I thought it would
> > > be more user-friendly to give a WARNING if the PUBLICATION's explicit
> > > column list includes generated cols when the option
> > > "publish_generated_columns" is false. This combination doesn't seem
> > > like something a user would do intentionally, so just silently
> > > ignoring it (like the current patch does) is likely going to give
> > > someone unexpected results/grief.
> > >
> > > ======
> > > src/backend/replication/logical/proto.c
> > >
> > > 4. logicalrep_write_tuple, and logicalrep_write_attrs:
> > >
> > > - if (att->attisdropped || att->attgenerated)
> > > + if (att->attisdropped)
> > > continue;
> > >
> > > Why aren't you also checking the new PUBLICATION option here and
> > > skipping all gencols if the "publish_generated_columns" option is
> > > false? Or is the BMS of pgoutput_column_list_init handling this case?
> > > Maybe there should be an Assert for this?
> > >
> > > ======
> > > src/backend/replication/pgoutput/pgoutput.c
> > >
> > > 5. send_relation_and_attrs
> > >
> > > - if (att->attisdropped || att->attgenerated)
> > > + if (att->attisdropped)
> > > continue;
> > >
> > > Same question as #4.
> > >
> > > ~~~
> > >
> > > 6. prepare_all_columns_bms and pgoutput_column_list_init
> > >
> > > + if (att->attgenerated && !pub->pubgencolumns)
> > > + cols = bms_del_member(cols, i + 1);
> > >
> > > IIUC, the algorithm seems overly tricky filling the BMS with all
> > > columns, before straight away conditionally removing the generated
> > > columns. Can't it be refactored to assign all the correct columns
> > > up-front, to avoid calling bms_del_member()?
> > >
> > > ======
> > > src/bin/pg_dump/pg_dump.c
> > >
> > > 7. getPublications
> > >
> > > IIUC, there is lots of missing SQL code here (for all older versions)
> > > that should be saying "false AS pubgencolumns".
> > > e.g. compare the SQL with how "false AS pubviaroot" is used.
> > >
> > > ======
> > > src/bin/pg_dump/t/002_pg_dump.pl
> > >
> > > 8. Missing tests?
> > >
> > > I expected to see a pg_dump test for this new PUBLICATION option.
> > >
> > > ======
> > > src/test/regress/sql/publication.sql
> > >
> > > 9. Missing tests?
> > >
> > > How about adding another test case that checks this new option must be
> > > "Boolean"?
> > >
> > > ~~~
> > >
> > > 10. Missing tests?
> > >
> > > --- error: generated column "d" can't be in list
> > > +-- ok: generated columns can be in the list too
> > > ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
> > > +ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
> > >
> > > (see my earlier comment #3)
> > >
> > > IMO there should be another test case for a WARNING here if the user
> > > attempts to include generated column 'd' in an explicit PUBLICATION
> > > column list while the "publish_generated-columns" is false.
> > >
> > > ======
> > > [1] https://www.postgresql.org/message-id/CAD21AoA-tdTz0G-vri8KM2TXeFU8RCDsOpBXUBCgwkfokF7%3DjA%40mail.g...
> > >
> >
> > I have fixed all the comments. The attached patches contain the desired changes.
> > Also the merging of 0001 and 0002 can be done once there are no
> > comments on the patch to help in reviewing.
>
> The warning message appears to be incorrect. Even though
> publish_generated_columns is set to true, the warning indicates that
> it is false.
> CREATE TABLE t1 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> postgres=# CREATE PUBLICATION pub1 FOR table t1(gen1) WITH
> (publish_generated_columns=true);
> WARNING: specified generated column "gen1" in publication column list
> for publication with publish_generated_columns as false
>
I have fixed the warning message. The attached patches contain the
desired changes.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v34-0003-DOCS-Generated-Column-Replication.patch (11.7K, ../../CAHv8RjJkUdYCdK_bL3yvEV=zKrA2dsnZYa1VMT2H5v0+qbaGbA@mail.gmail.com/2-v34-0003-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From b7cc820c516675de0e54561961fcdacb59d621c4 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Fri, 27 Sep 2024 19:31:55 +0530
Subject: [PATCH v34 3/3] DOCS - Generated Column Replication.
This patch adds a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By:
Discussion:
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 269 +++++++++++++++++++++++
doc/src/sgml/ref/create_publication.sgml | 4 +
3 files changed, 275 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 7b9c349343..192180d658 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> parameter
- <link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>publish_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..22d378bc6c 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1567,6 +1567,275 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index f9ecdeefb9..2119262cbc 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -235,6 +235,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
set to <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.41.0.windows.3
[application/octet-stream] v34-0001-Enable-support-for-publish_generated_columns-opt.patch (83.0K, ../../CAHv8RjJkUdYCdK_bL3yvEV=zKrA2dsnZYa1VMT2H5v0+qbaGbA@mail.gmail.com/3-v34-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 41975a87537549477978bc3df7dcc16f913a0cee Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Fri, 23 Aug 2024 10:03:38 +1000
Subject: [PATCH v34 1/3] Enable support for 'publish_generated_columns'
option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit supports the transmission of generated column information and data
alongside regular table changes. This behaviour is controlled by a new
PUBLICATION parameter ('publish_generated_columns').
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
When 'publish_generated_columns' is false, generated columns are not replicated.
But when generated columns are specified in PUBLICATION col-list, it is
replicated even the 'publish_generated_columns' is false.
There is a change in 'pg_publicataion' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 17 +-
src/backend/commands/publicationcmds.c | 36 +-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/pgoutput/pgoutput.c | 102 +++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/pg_publication.h | 7 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 478 +++++++++++---------
src/test/regress/sql/publication.sql | 38 +-
src/test/subscription/t/031_column_list.pl | 4 +-
17 files changed, 491 insertions(+), 277 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..7b9c349343 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 11b6456779..56de72c0c6 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6543,7 +6543,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2b86..583da09748 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -420,7 +420,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pub->pubgencols);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -507,7 +508,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
{
Bitmapset *set = NULL;
ListCell *lc;
@@ -530,10 +531,10 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated && !pubgencols)
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list when publish_generated_columns as false",
colname));
if (bms_is_member(attnum, set))
@@ -1006,6 +1007,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1216,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..8c09125170 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
@@ -1182,7 +1209,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..24c56ed894 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,36 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Prepare new column list bitmap.
+ * This encompasses all table columns, excluding the generated ones.
+ */
+static Bitmapset *
+prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,13 +1072,14 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * To handle cases where the publish_generated_columns option isn't
+ * specified for all tables in a publication, the pubgencolumns check
+ * needs to be performed. In such cases, we must create a column list
+ * that excludes generated columns.
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = true;
/*
* Check for the presence of a column list in this publication.
@@ -1066,44 +1097,47 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
- &pub_no_list);
+ &pub_rel_has_collist);
+ }
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ /* Build the column list bitmap in the per-entry context. */
+ if (!pub_rel_has_collist || !pub->pubgencols)
+ {
+ int i;
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- pgoutput_ensure_entry_cxt(data, entry);
+ pgoutput_ensure_entry_cxt(data, entry);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ if (!pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_all_columns_bms(data, entry, desc);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ /* Get the number of live attributes. */
+ for (i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
+ if (att->attisdropped)
+ continue;
- nliveatts++;
- }
+ nliveatts++;
+ }
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
}
+ }
+ if (HeapTupleIsValid(cftuple))
ReleaseSysCache(cftuple);
- }
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 130b80775d..a0dad1ebf7 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4293,23 +4294,29 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4326,6 +4333,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4350,6 +4358,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4429,6 +4439,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91083..16cbef3693 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a7ccde6d7d..ea36b18ea2 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3182,7 +3182,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..2a3816f661 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -152,7 +156,8 @@ extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool pubgencols);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..a26b5a418e 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,10 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+WARNING: specified generated column "d" in publication column list when publish_generated_columns as false
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +737,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +924,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1132,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1173,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1254,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1267,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1296,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1322,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1393,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1404,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1425,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1437,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1449,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1460,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1471,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1482,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1513,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1525,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1607,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1628,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1756,53 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- gencols in column list with 'publish_generated_columns'=true.
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+-- No gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- ALTER PUBLICATION to add gencols to column list.
+ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP PUBLICATION pub3;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..d7b43dcf68 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1110,6 +1113,39 @@ DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- gencols in column list with 'publish_generated_columns'=true.
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+-- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- No gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
+-- ALTER PUBLICATION to add gencols to column list.
+ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP PUBLICATION pub3;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.41.0.windows.3
[application/octet-stream] v34-0002-Support-replication-of-generated-column-during-i.patch (18.3K, ../../CAHv8RjJkUdYCdK_bL3yvEV=zKrA2dsnZYa1VMT2H5v0+qbaGbA@mail.gmail.com/4-v34-0002-Support-replication-of-generated-column-during-i.patch)
download | inline diff:
From 184107e6352efccbb650d85ed0c640bd7ebc88e6 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Tue, 24 Sep 2024 14:44:44 +0530
Subject: [PATCH v34 2/3] Support replication of generated column during
initial sync
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
when (publish_generated_columns = false)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column:
Publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* publisher generated column => subscriber generated column: Publisher
generated column is not replicated. The subscriber generated column
will be filed with the subscriber-side computed or default data.
---
doc/src/sgml/ref/create_publication.sgml | 4 +
src/backend/catalog/pg_subscription.c | 31 +++
src/backend/commands/subscriptioncmds.c | 31 ---
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 197 ++++++++++++++++----
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
7 files changed, 203 insertions(+), 69 deletions(-)
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..f9ecdeefb9 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
+ set to <literal>false</literal>.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc636b8..addf307cb6 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -439,37 +439,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
-/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
/*
* Check that the specified publications are present on the publisher.
*/
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..22ebe40336 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,24 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +930,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +986,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1034,28 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1067,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -1005,6 +1100,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1015,7 +1113,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1135,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1221,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1236,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1292,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1363,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..158b444275 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
--
2.41.0.windows.3
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-30 06:17 ` Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-30 06:17 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Here are my review comments for patch v34-0001
======
doc/src/sgml/ddl.sgml
1.
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication
according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
This information is not quite correct because it makes no mention of
PUBLICATION column lists. OTOH I replaced all this paragraph in the
0003 patch anyhow, so maybe it is not worth worrying about this review
comment.
======
src/backend/catalog/pg_publication.c
2.
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated && !pubgencols)
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list
when publish_generated_columns as false",
colname));
Back when I proposed to have this WARNING I don't think there existed
the idea that the PUBLICATION column list would override the
'publish_generated_columns' parameter. But now that this is the
implementation, I am no longer 100% sure if a warning should be given
at all because this can be a perfectly valid combination. What do
others think?
======
src/backend/replication/pgoutput/pgoutput.c
3. prepare_all_columns_bms
3a.
nit - minor rewording function comment
~
3b.
I am not sure this is a good function name, particularly since it does
not return "all" columns. Can you name it more for what it does?
~~~
4. pgoutput_column_list_init
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * To handle cases where the publish_generated_columns option isn't
+ * specified for all tables in a publication, the pubgencolumns check
+ * needs to be performed. In such cases, we must create a column list
+ * that excludes generated columns.
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
4a.
That comment still doesn't make much sense to me:
e.g.1. "To handle cases where the publish_generated_columns option
isn't specified for all tables in a publication". What is this trying
to say? A publication parameter is per-publication, so it is always
"for all tables in a publication".
e.g.2. " the pubgencolumns check" -- what is the pubgencols check?
Please rewrite the comment more clearly to explain the logic: What is
it doing? Why is it doing it?
~
4b.
+ if (!pub->alltables || !pub->pubgencols)
As mentioned in a previous review, there is too much negativity in
conditions like this. I think anything you can do to reverse all the
negativity will surely improve the readability of this function. See
[1 - #11a]
~
5.
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ if (!pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_all_columns_bms(data, entry, desc);
Hm. Is that correct? The if/else is all backwards from what I would
have expected. IIUC the variable 'pub_rel_has_collist' is assigned to
the opposite value of what the name says, so then you are using it all
backwards to make it work.
nit - I have changed the code in the attachment how I think it should
be. Please check it makes sense.
~
6.
+ /* Get the number of live attributes. */
+ for (i = 0; i < desc->natts; i++)
nit - use a for-loop variable 'i'.
======
src/bin/pg_dump/pg_dump.c
7.
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
p.pubtruncate, p.pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS
pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS
pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
These changes are all wrong due to a typo. There is no such column as
'pubviagencols'. I made these changes in the nit attachment. Please
check it for correctness.
s/pubviagencols/pubgencols/
======
src/test/regress/sql/publication.sql
8.
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
nit - name the generated column "d", to clarify this comment
~~~
9.
+-- Test the publication 'publish_generated_columns' parameter enabled
or disabled for different combinations.
nit - add another "======" separator comment before the new test
nit - some minor adjustments to whitespace and comments
~~~
10.
+-- No gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
+-- ALTER PUBLICATION to add gencols to column list.
+ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
nit - by adding and removing collist for the existing table, we don't
need to have a 'pub3'.
======
src/test/subscription/t/031_column_list.pl
11.
It is not clear to me -- is there, or is there not yet any test case
for the multiple publication issues that were discussed previously?
e.g. when the same table has gencols but there are multiple
publications for the same subscription and the
'publish_generated_columns' parameter or column lists conflict.
======
[1] https://www.postgresql.org/message-id/CAHut%2BPt9ArtYc1Vtb1DqzEEFwjcoDHMyRCnUqkHQeFJuMCDkvQ%40mail.g...
Kind Regards,
Peter Smith.
Fujitsu Australia
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 24c56ed..da7ebe9 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1009,8 +1009,9 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
/*
- * Prepare new column list bitmap.
- * This encompasses all table columns, excluding the generated ones.
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
*/
static Bitmapset *
prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
@@ -1079,7 +1080,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
*/
if (!pub->alltables || !pub->pubgencols)
{
- bool pub_rel_has_collist = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1094,28 +1095,31 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
- &pub_rel_has_collist);
+ &pub_no_list);
+
+ pub_rel_has_collist = !pub_no_list;
}
/* Build the column list bitmap in the per-entry context. */
- if (!pub_rel_has_collist || !pub->pubgencols)
+ if (pub_rel_has_collist || !pub->pubgencols)
{
- int i;
int nliveatts = 0;
TupleDesc desc = RelationGetDescr(relation);
pgoutput_ensure_entry_cxt(data, entry);
- if (!pub_rel_has_collist)
+ if (pub_rel_has_collist)
cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
else
cols = prepare_all_columns_bms(data, entry, desc);
/* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
+ for (int i = 0; i < desc->natts; i++)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a0dad1e..ec12fd9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4304,19 +4304,19 @@ getPublications(Archive *fout)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index a26b5a4..a532cd6 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -693,7 +693,7 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- ok: generated columns can be in the list too
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
WARNING: specified generated column "d" in publication column list when publish_generated_columns as false
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
@@ -1756,6 +1756,7 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
-- Test the publication 'publish_generated_columns' parameter enabled or disabled
SET client_min_messages = 'ERROR';
CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
@@ -1777,30 +1778,29 @@ CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
--- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false.
+-- gencols in column list with 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
WARNING: "wal_level" is insufficient to publish logical changes
HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list with 'publish_generated_columns'=true.
+-- gencols in column list with 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
WARNING: "wal_level" is insufficient to publish logical changes
HINT: Set "wal_level" to "logical" before creating subscriptions.
--- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+-- gencols in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- No gencols in column list with 'publish_generated_columns'=false.
-CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- ALTER PUBLICATION to add gencols to column list.
-ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-DROP PUBLICATION pub3;
DROP TABLE gencols;
RESET client_min_messages;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index d7b43dc..6a74fd6 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -415,7 +415,7 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- ok: generated columns can be in the list too
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
@@ -1112,6 +1112,7 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
-- Test the publication 'publish_generated_columns' parameter enabled or disabled
SET client_min_messages = 'ERROR';
@@ -1125,24 +1126,28 @@ RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
--- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false.
+
+-- gencols in column list with 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
--- gencols in column list with 'publish_generated_columns'=true.
+
+-- gencols in column list with 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
--- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+
+-- gencols in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- No gencols in column list with 'publish_generated_columns'=false.
-CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
--- ALTER PUBLICATION to add gencols to column list.
-ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-DROP PUBLICATION pub3;
DROP TABLE gencols;
RESET client_min_messages;
Attachments:
[text/plain] PS_NITPICKS_20240930_V340001.txt (9.9K, ../../CAHut+Ps8wcnBHoDf0y8wQKoP=Uk7imQ7_X6HNfvr-KLqLe6C1g@mail.gmail.com/2-PS_NITPICKS_20240930_V340001.txt)
download | inline diff:
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 24c56ed..da7ebe9 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1009,8 +1009,9 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
/*
- * Prepare new column list bitmap.
- * This encompasses all table columns, excluding the generated ones.
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
*/
static Bitmapset *
prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
@@ -1079,7 +1080,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
*/
if (!pub->alltables || !pub->pubgencols)
{
- bool pub_rel_has_collist = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1094,28 +1095,31 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
- &pub_rel_has_collist);
+ &pub_no_list);
+
+ pub_rel_has_collist = !pub_no_list;
}
/* Build the column list bitmap in the per-entry context. */
- if (!pub_rel_has_collist || !pub->pubgencols)
+ if (pub_rel_has_collist || !pub->pubgencols)
{
- int i;
int nliveatts = 0;
TupleDesc desc = RelationGetDescr(relation);
pgoutput_ensure_entry_cxt(data, entry);
- if (!pub_rel_has_collist)
+ if (pub_rel_has_collist)
cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
else
cols = prepare_all_columns_bms(data, entry, desc);
/* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
+ for (int i = 0; i < desc->natts; i++)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a0dad1e..ec12fd9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4304,19 +4304,19 @@ getPublications(Archive *fout)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubviagencols "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index a26b5a4..a532cd6 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -693,7 +693,7 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- ok: generated columns can be in the list too
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
WARNING: specified generated column "d" in publication column list when publish_generated_columns as false
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
@@ -1756,6 +1756,7 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
-- Test the publication 'publish_generated_columns' parameter enabled or disabled
SET client_min_messages = 'ERROR';
CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
@@ -1777,30 +1778,29 @@ CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
--- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false.
+-- gencols in column list with 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
WARNING: "wal_level" is insufficient to publish logical changes
HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list with 'publish_generated_columns'=true.
+-- gencols in column list with 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
WARNING: "wal_level" is insufficient to publish logical changes
HINT: Set "wal_level" to "logical" before creating subscriptions.
--- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+-- gencols in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- No gencols in column list with 'publish_generated_columns'=false.
-CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- ALTER PUBLICATION to add gencols to column list.
-ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-DROP PUBLICATION pub3;
DROP TABLE gencols;
RESET client_min_messages;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index d7b43dc..6a74fd6 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -415,7 +415,7 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- ok: generated columns can be in the list too
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
@@ -1112,6 +1112,7 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
-- Test the publication 'publish_generated_columns' parameter enabled or disabled
SET client_min_messages = 'ERROR';
@@ -1125,24 +1126,28 @@ RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
--- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false.
+
+-- gencols in column list with 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
--- gencols in column list with 'publish_generated_columns'=true.
+
+-- gencols in column list with 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
--- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+
+-- gencols in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- No gencols in column list with 'publish_generated_columns'=false.
-CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
--- ALTER PUBLICATION to add gencols to column list.
-ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-DROP PUBLICATION pub3;
DROP TABLE gencols;
RESET client_min_messages;
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-03 11:00 ` Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-04 04:12 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 2 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-03 11:00 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Sep 30, 2024 at 11:47 AM Peter Smith <[email protected]> wrote:
>
> Here are my review comments for patch v34-0001
>
> ======
> doc/src/sgml/ddl.sgml
>
> 1.
> - Generated columns are skipped for logical replication and cannot be
> - specified in a <command>CREATE PUBLICATION</command> column list.
> + Generated columns may be skipped during logical replication
> according to the
> + <command>CREATE PUBLICATION</command> parameter
> + <link linkend="sql-createpublication-params-with-publish-generated-columns">
> + <literal>publish_generated_columns</literal></link>.
>
> This information is not quite correct because it makes no mention of
> PUBLICATION column lists. OTOH I replaced all this paragraph in the
> 0003 patch anyhow, so maybe it is not worth worrying about this review
> comment.
>
> ======
> src/backend/catalog/pg_publication.c
>
> 2.
> - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> - ereport(ERROR,
> + if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated && !pubgencols)
> + ereport(WARNING,
> errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> - errmsg("cannot use generated column \"%s\" in publication column list",
> + errmsg("specified generated column \"%s\" in publication column list
> when publish_generated_columns as false",
> colname));
>
> Back when I proposed to have this WARNING I don't think there existed
> the idea that the PUBLICATION column list would override the
> 'publish_generated_columns' parameter. But now that this is the
> implementation, I am no longer 100% sure if a warning should be given
> at all because this can be a perfectly valid combination. What do
> others think?
>
> ======
> src/backend/replication/pgoutput/pgoutput.c
>
> 3. prepare_all_columns_bms
>
> 3a.
> nit - minor rewording function comment
>
> ~
>
> 3b.
> I am not sure this is a good function name, particularly since it does
> not return "all" columns. Can you name it more for what it does?
>
> ~~~
>
> 4. pgoutput_column_list_init
>
> /*
> - * If the publication is FOR ALL TABLES then it is treated the same as
> - * if there are no column lists (even if other publications have a
> - * list).
> + * To handle cases where the publish_generated_columns option isn't
> + * specified for all tables in a publication, the pubgencolumns check
> + * needs to be performed. In such cases, we must create a column list
> + * that excludes generated columns.
> */
> - if (!pub->alltables)
> + if (!pub->alltables || !pub->pubgencols)
>
> 4a.
> That comment still doesn't make much sense to me:
>
> e.g.1. "To handle cases where the publish_generated_columns option
> isn't specified for all tables in a publication". What is this trying
> to say? A publication parameter is per-publication, so it is always
> "for all tables in a publication".
>
> e.g.2. " the pubgencolumns check" -- what is the pubgencols check?
>
> Please rewrite the comment more clearly to explain the logic: What is
> it doing? Why is it doing it?
>
> ~
>
> 4b.
> + if (!pub->alltables || !pub->pubgencols)
>
> As mentioned in a previous review, there is too much negativity in
> conditions like this. I think anything you can do to reverse all the
> negativity will surely improve the readability of this function. See
> [1 - #11a]
>
>
> ~
>
> 5.
> - cols = pub_collist_to_bitmapset(cols, cfdatum,
> - entry->entry_cxt);
> + if (!pub_rel_has_collist)
> + cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
> + else
> + cols = prepare_all_columns_bms(data, entry, desc);
>
> Hm. Is that correct? The if/else is all backwards from what I would
> have expected. IIUC the variable 'pub_rel_has_collist' is assigned to
> the opposite value of what the name says, so then you are using it all
> backwards to make it work.
>
> nit - I have changed the code in the attachment how I think it should
> be. Please check it makes sense.
>
> ~
>
> 6.
> + /* Get the number of live attributes. */
> + for (i = 0; i < desc->natts; i++)
>
> nit - use a for-loop variable 'i'.
>
> ======
> src/bin/pg_dump/pg_dump.c
>
> 7.
> + else if (fout->remoteVersion >= 130000)
> + appendPQExpBufferStr(query,
> + "SELECT p.tableoid, p.oid, p.pubname, "
> + "p.pubowner, "
> + "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
> p.pubtruncate, p.pubviaroot, false AS pubviagencols "
> "FROM pg_publication p");
> else if (fout->remoteVersion >= 110000)
> appendPQExpBufferStr(query,
> "SELECT p.tableoid, p.oid, p.pubname, "
> "p.pubowner, "
> - "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
> p.pubtruncate, false AS pubviaroot "
> + "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete,
> p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
> "FROM pg_publication p");
> else
> appendPQExpBufferStr(query,
> "SELECT p.tableoid, p.oid, p.pubname, "
> "p.pubowner, "
> - "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS
> pubtruncate, false AS pubviaroot "
> + "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS
> pubtruncate, false AS pubviaroot, false AS pubviagencols "
> "FROM pg_publication p");
>
> These changes are all wrong due to a typo. There is no such column as
> 'pubviagencols'. I made these changes in the nit attachment. Please
> check it for correctness.
>
> s/pubviagencols/pubgencols/
>
> ======
> src/test/regress/sql/publication.sql
>
> 8.
> --- error: generated column "d" can't be in list
> +-- ok: generated columns can be in the list too
>
> nit - name the generated column "d", to clarify this comment
>
> ~~~
>
> 9.
> +-- Test the publication 'publish_generated_columns' parameter enabled
> or disabled for different combinations.
>
> nit - add another "======" separator comment before the new test
> nit - some minor adjustments to whitespace and comments
>
> ~~~
>
> 10.
> +-- No gencols in column list with 'publish_generated_columns'=false.
> +CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
> +-- ALTER PUBLICATION to add gencols to column list.
> +ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
>
> nit - by adding and removing collist for the existing table, we don't
> need to have a 'pub3'.
>
> ======
> src/test/subscription/t/031_column_list.pl
>
> 11.
> It is not clear to me -- is there, or is there not yet any test case
> for the multiple publication issues that were discussed previously?
> e.g. when the same table has gencols but there are multiple
> publications for the same subscription and the
> 'publish_generated_columns' parameter or column lists conflict.
>
I have moved the new test cases to the 011_generated.pl file and
created a separate patch . I will post the TAP-TESTS once these
patches get fixed.
I have fixed all the given comments. The attached v36-0001 patch
contain the desired changes.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v36-0002-Support-replication-of-generated-column-during-i.patch (17.5K, ../../CAHv8Rj+1RDd7AnJNzOJXk--zcbTtU3nys=ZgU3ktB4e3DWbJgg@mail.gmail.com/2-v36-0002-Support-replication-of-generated-column-during-i.patch)
download | inline diff:
From 3962cc7928453df966f1bb1969f70a1194e627aa Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Tue, 24 Sep 2024 14:44:44 +0530
Subject: [PATCH v36 2/3] Support replication of generated column during
initial sync
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
when (publish_generated_columns = false)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column:
Publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* publisher generated column => subscriber generated column: Publisher
generated column is not replicated. The subscriber generated column
will be filed with the subscriber-side computed or default data.
---
src/backend/catalog/pg_subscription.c | 31 +++
src/backend/commands/subscriptioncmds.c | 31 ---
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 197 ++++++++++++++++----
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
6 files changed, 199 insertions(+), 69 deletions(-)
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc636b8..addf307cb6 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -439,37 +439,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
-/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
/*
* Check that the specified publications are present on the publisher.
*/
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..22ebe40336 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,24 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +930,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +986,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1034,28 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1067,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -1005,6 +1100,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1015,7 +1113,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1135,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1221,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1236,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1292,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1363,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..158b444275 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
--
2.41.0.windows.3
[application/octet-stream] v36-0003-DOCS-Generated-Column-Replication.patch (12.4K, ../../CAHv8Rj+1RDd7AnJNzOJXk--zcbTtU3nys=ZgU3ktB4e3DWbJgg@mail.gmail.com/3-v36-0003-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From ea8658755cb361de76009ca624b408a2e6c84fcd Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 1 Oct 2024 12:07:56 +0530
Subject: [PATCH v36 3/3] DOCS - Generated Column Replication.
This patch adds a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By:
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 277 +++++++++++++++++++++++
doc/src/sgml/ref/create_publication.sgml | 4 +
3 files changed, 283 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 7b9c349343..192180d658 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> parameter
- <link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>publish_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..8e80a3ea84 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,275 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.41.0.windows.3
[application/octet-stream] v36-0001-Enable-support-for-publish_generated_columns-opt.patch (82.7K, ../../CAHv8Rj+1RDd7AnJNzOJXk--zcbTtU3nys=ZgU3ktB4e3DWbJgg@mail.gmail.com/4-v36-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From b0db19a222341f180b3d27fcafd0f752f7559135 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Fri, 23 Aug 2024 10:03:38 +1000
Subject: [PATCH v36] Enable support for 'publish_generated_columns' option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit supports the transmission of generated column information and data
alongside regular table changes. This behaviour is controlled by a new
PUBLICATION parameter ('publish_generated_columns').
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
When 'publish_generated_columns' is false, generated columns are not replicated.
But when generated columns are specified in PUBLICATION col-list, it is
replicated even the 'publish_generated_columns' is false.
There is a change in 'pg_publicataion' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 18 +-
src/backend/commands/publicationcmds.c | 36 +-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/pgoutput/pgoutput.c | 104 +++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/pg_publication.h | 7 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 474 +++++++++++---------
src/test/regress/sql/publication.sql | 43 +-
src/test/subscription/t/031_column_list.pl | 4 +-
17 files changed, 492 insertions(+), 280 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..7b9c349343 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 2d2481bb8b..c1b9b62505 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6543,7 +6543,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2b86..7eed2a9d10 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -420,7 +420,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pub->pubgencols);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -507,11 +508,10 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +530,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +1000,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1209,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..8c09125170 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
@@ -1182,7 +1209,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..18e3f55943 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,37 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
+ */
+static Bitmapset *
+prepare_nogen_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,13 +1073,14 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * To handle cases where the publish_generated_columns option isn't
+ * specified for all tables in a publication, we must create a column
+ * list that excludes generated columns. So, the publisher will not
+ * replicate the generated columns.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1063,47 +1095,53 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ pub_rel_has_collist = !pub_no_list;
+ }
- pgoutput_ensure_entry_cxt(data, entry);
+ /* Build the column list bitmap in the per-entry context. */
+ if (pub_rel_has_collist || !pub->pubgencols)
+ {
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ pgoutput_ensure_entry_cxt(data, entry);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_nogen_columns_bms(data, entry, desc);
- if (att->attisdropped || att->attgenerated)
- continue;
+ /* Get the number of live attributes. */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- nliveatts++;
- }
+ if (att->attisdropped)
+ continue;
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ nliveatts++;
}
- ReleaseSysCache(cftuple);
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
+ }
}
+
+ if (HeapTupleIsValid(cftuple))
+ ReleaseSysCache(cftuple);
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 130b80775d..ec12fd9715 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4293,23 +4294,29 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4326,6 +4333,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4350,6 +4358,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4429,6 +4439,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91083..16cbef3693 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a7ccde6d7d..ea36b18ea2 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3182,7 +3182,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..2a3816f661 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -152,7 +156,8 @@ extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool pubgencols);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..8bcb79ad42 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,50 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- gencols in column list with 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- gencols in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..6a74fd6229 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,45 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+
+-- gencols in column list with 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+
+-- gencols in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- remove gencols from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add gencols in column list, when 'publish_generated_columns'=false.
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.34.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-04 04:05 ` Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-10-04 04:05 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi Shubham, here are my review comments for v36-0001.
======
1. General - merge patches
It is long past due when patches 0001 and 0002 should've been merged.
AFAIK the split was only because historically these parts had
different authors. But, keeping them separated is not helpful anymore.
======
src/backend/catalog/pg_publication.c
2.
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
Since you removed the WARNING, this parameter 'pubgencols' is unused
so it should also be removed.
======
src/backend/replication/pgoutput/pgoutput.c
3.
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * To handle cases where the publish_generated_columns option isn't
+ * specified for all tables in a publication, we must create a column
+ * list that excludes generated columns. So, the publisher will not
+ * replicate the generated columns.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
I still found that comment hard to understand. Does this mean to say
something like:
------
Process potential column lists for the following cases:
a. Any publication that is not FOR ALL TABLES.
b. When the publication is FOR ALL TABLES and
'publish_generated_columns' is false.
A FOR ALL TABLES publication doesn't have user-defined column lists,
so all columns will be replicated by default. However, if
'publish_generated_columns' is set to false, column lists must still
be created to exclude any generated columns from being published
------
======
src/test/regress/sql/publication.sql
4.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
AFAIK you don't need to keep changing 'client_min_messages',
particularly now that you've removed the WARNING message that was
previously emitted.
~
5.
nit - minor comment changes.
======
Please refer to the attachment which implements any nits from above.
======
Kind Regards,
Peter Smith.
Fujitsu Austrlia.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 8bcb79a..e419ca8 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -1774,26 +1774,20 @@ CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
regress_publication_user | t | t | t | t | t | f | f
(1 row)
-RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-- Test the 'publish_generated_columns' parameter enabled or disabled for
-- different scenarios with/without generated columns in column lists.
-SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false
+-- Generated columns in column list, when 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list with 'publish_generated_columns'=true
+-- Generated columns in column list, when 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list, then set 'publication_generate_columns'=false
+-- Generated columns in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- remove gencols from column list, when 'publish_generated_columns'=false
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
--- Add gencols in column list, when 'publish_generated_columns'=false.
+-- Add generated columns in column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 6a74fd6..9724ba3 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -1118,32 +1118,29 @@ DROP SCHEMA sch2 cascade;
SET client_min_messages = 'ERROR';
CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
\dRp+ pub1
-
CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
\dRp+ pub2
-RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-- Test the 'publish_generated_columns' parameter enabled or disabled for
-- different scenarios with/without generated columns in column lists.
-SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false
+-- Generated columns in column list, when 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
--- gencols in column list with 'publish_generated_columns'=true
+-- Generated columns in column list, when 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
--- gencols in column list, then set 'publication_generate_columns'=false
+-- Generated columns in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- remove gencols from column list, when 'publish_generated_columns'=false
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
--- Add gencols in column list, when 'publish_generated_columns'=false.
+-- Add generated columns in column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
Attachments:
[text/plain] PS_NITPICKS_GENCOLS_v360001.txt (4.0K, ../../CAHut+Ps9ksUQsRNWJtAj-SrMqfQq4pixfA2DSNOU3+nk_hPmTg@mail.gmail.com/2-PS_NITPICKS_GENCOLS_v360001.txt)
download | inline diff:
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 8bcb79a..e419ca8 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -1774,26 +1774,20 @@ CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
regress_publication_user | t | t | t | t | t | f | f
(1 row)
-RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-- Test the 'publish_generated_columns' parameter enabled or disabled for
-- different scenarios with/without generated columns in column lists.
-SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false
+-- Generated columns in column list, when 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list with 'publish_generated_columns'=true
+-- Generated columns in column list, when 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
-WARNING: "wal_level" is insufficient to publish logical changes
-HINT: Set "wal_level" to "logical" before creating subscriptions.
--- gencols in column list, then set 'publication_generate_columns'=false
+-- Generated columns in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- remove gencols from column list, when 'publish_generated_columns'=false
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
--- Add gencols in column list, when 'publish_generated_columns'=false.
+-- Add generated columns in column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 6a74fd6..9724ba3 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -1118,32 +1118,29 @@ DROP SCHEMA sch2 cascade;
SET client_min_messages = 'ERROR';
CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
\dRp+ pub1
-
CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
\dRp+ pub2
-RESET client_min_messages;
DROP PUBLICATION pub1;
DROP PUBLICATION pub2;
-- Test the 'publish_generated_columns' parameter enabled or disabled for
-- different scenarios with/without generated columns in column lists.
-SET client_min_messages = 'WARNING';
CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
--- gencols in column list with 'publish_generated_columns'=false
+-- Generated columns in column list, when 'publish_generated_columns'=false
CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
--- gencols in column list with 'publish_generated_columns'=true
+-- Generated columns in column list, when 'publish_generated_columns'=true
CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
--- gencols in column list, then set 'publication_generate_columns'=false
+-- Generated columns in column list, then set 'publication_generate_columns'=false
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
--- remove gencols from column list, when 'publish_generated_columns'=false
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
--- Add gencols in column list, when 'publish_generated_columns'=false.
+-- Add generated columns in column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
DROP PUBLICATION pub1;
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-08 06:07 ` Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-09 05:30 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-09 06:22 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-09 09:23 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-10 05:23 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 6 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-08 06:07 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
>
> Hi Shubham, here are my review comments for v36-0001.
>
> ======
> 1. General - merge patches
>
> It is long past due when patches 0001 and 0002 should've been merged.
> AFAIK the split was only because historically these parts had
> different authors. But, keeping them separated is not helpful anymore.
>
> ======
> src/backend/catalog/pg_publication.c
>
> 2.
> Bitmapset *
> -pub_collist_validate(Relation targetrel, List *columns)
> +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
>
> Since you removed the WARNING, this parameter 'pubgencols' is unused
> so it should also be removed.
>
> ======
> src/backend/replication/pgoutput/pgoutput.c
>
> 3.
> /*
> - * If the publication is FOR ALL TABLES then it is treated the same as
> - * if there are no column lists (even if other publications have a
> - * list).
> + * To handle cases where the publish_generated_columns option isn't
> + * specified for all tables in a publication, we must create a column
> + * list that excludes generated columns. So, the publisher will not
> + * replicate the generated columns.
> */
> - if (!pub->alltables)
> + if (!(pub->alltables && pub->pubgencols))
>
> I still found that comment hard to understand. Does this mean to say
> something like:
>
> ------
> Process potential column lists for the following cases:
>
> a. Any publication that is not FOR ALL TABLES.
>
> b. When the publication is FOR ALL TABLES and
> 'publish_generated_columns' is false.
> A FOR ALL TABLES publication doesn't have user-defined column lists,
> so all columns will be replicated by default. However, if
> 'publish_generated_columns' is set to false, column lists must still
> be created to exclude any generated columns from being published
> ------
>
> ======
> src/test/regress/sql/publication.sql
>
> 4.
> +SET client_min_messages = 'WARNING';
> +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
>
> AFAIK you don't need to keep changing 'client_min_messages',
> particularly now that you've removed the WARNING message that was
> previously emitted.
>
> ~
>
> 5.
> nit - minor comment changes.
>
> ======
> Please refer to the attachment which implements any nits from above.
>
I have fixed all the given comments. Also, I have created a new 0003
patch for the TAP-Tests related to the '011_generated.pl' file. I am
planning to merge 0001 and 0003 patches once they will get fixed.
The attached patches contain the required changes.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v37-0003-Tap-tests-for-generated-columns.patch (14.1K, ../../CAHv8Rj+Rnw+_SfSyyrvWL49AfJzx4O8YVvdU9gB+SQdt3=qF+A@mail.gmail.com/2-v37-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 46bf6a00617e1434de0277f899a37bc17bb5b6d8 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Fri, 23 Aug 2024 20:39:57 +1000
Subject: [PATCH v37 3/3] Tap tests for generated columns
Add tests for all combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna, Peter Smith
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 318 +++++++++++++++++++++++
1 file changed, 318 insertions(+)
mode change 100644 => 100755 src/test/subscription/t/011_generated.pl
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
old mode 100644
new mode 100755
index 8b2e5f4708..5dd5965da7
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,322 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for all combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, all combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription with copy_data=false.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = false);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+# XXX copy_data=false for now. This will be changed later.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is($result, qq(),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# Test: no column list, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO gen_to_nogen VALUES (1, 1);
+ INSERT INTO gen_to_nogen2 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen(a, b, gen2), gen_to_nogen2 WITH (publish_generated_columns=false);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen2 ORDER BY c");
+is($result, qq(1|1||),
+ 'gen_to_nogen2 initial sync, when publish_generated_columns=false');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data excluding generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO nogen_to_gen VALUES (1, 1);
+ INSERT INTO nogen_to_gen2 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen, nogen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen ORDER BY a");
+is($result, qq(1|1||),
+ 'nogen_to_gen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen2 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO gen_to_nogen3 VALUES (1, 1);
+ INSERT INTO gen_to_nogen4 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen3(a, b, gen2), gen_to_nogen4 WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen3 ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen4 ORDER BY c");
+is($result, qq(1|1|2|2),
+ 'gen_to_nogen4 initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher publishes only the data of the columns specified in
+# column list skipping other generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen3 (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO nogen_to_gen3 VALUES (1, 1);
+ INSERT INTO nogen_to_gen4 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen3, nogen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen3 (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen3 ORDER BY a");
+is($result, qq(1|1|2|2),
+ 'nogen_to_gen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen4 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher replicates all columns if publish_generated_columns is
+# enabled and there is no column list
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_nogen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO nogen_to_nogen VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_nogen WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_nogen (a int, b int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_nogen ORDER BY a");
+is($result, qq(1|1|2|2),
+ 'nogen_to_nogen initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.41.0.windows.3
[application/octet-stream] v37-0002-DOCS-Generated-Column-Replication.patch (12.4K, ../../CAHv8Rj+Rnw+_SfSyyrvWL49AfJzx4O8YVvdU9gB+SQdt3=qF+A@mail.gmail.com/3-v37-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From 908f0707bee2b55b0700f6fab8d9a61032d83fce Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 1 Oct 2024 12:07:56 +0530
Subject: [PATCH v37 2/3] DOCS - Generated Column Replication.
This patch adds a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By:
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 277 +++++++++++++++++++++++
doc/src/sgml/ref/create_publication.sgml | 4 +
3 files changed, 283 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 7b9c349343..192180d658 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> parameter
- <link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>publish_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..8e80a3ea84 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,275 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.41.0.windows.3
[application/octet-stream] v37-0001-Enable-support-for-publish_generated_columns.patch (98.6K, ../../CAHv8Rj+Rnw+_SfSyyrvWL49AfJzx4O8YVvdU9gB+SQdt3=qF+A@mail.gmail.com/4-v37-0001-Enable-support-for-publish_generated_columns.patch)
download | inline diff:
From f0aef83bb03ca5cfb5faf45afa1ced5f0747c867 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 8 Oct 2024 11:02:36 +0530
Subject: [PATCH v37] Enable support for 'publish_generated_columns' option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit supports the transmission of generated column information and data
alongside regular table changes. This behaviour is controlled by a new
PUBLICATION parameter ('publish_generated_columns').
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
When 'publish_generated_columns' is false, generated columns are not replicated.
But when generated columns are specified in PUBLICATION col-list, it is
replicated even the 'publish_generated_columns' is false.
The replication of generated column during initial sync using tablesync:
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
when (publish_generated_columns = false)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column:
Publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* publisher generated column => subscriber generated column: Publisher
generated column is not replicated. The subscriber generated column
will be filed with the subscriber-side computed or default data.
There is a change in 'pg_publicataion' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 13 +-
src/backend/catalog/pg_subscription.c | 31 ++
src/backend/commands/publicationcmds.c | 33 +-
src/backend/commands/subscriptioncmds.c | 31 --
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 205 +++++++--
src/backend/replication/pgoutput/pgoutput.c | 108 +++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.in.c | 2 +-
src/include/catalog/pg_publication.h | 4 +
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 468 +++++++++++---------
src/test/regress/sql/publication.sql | 40 +-
src/test/subscription/t/031_column_list.pl | 4 +-
23 files changed, 687 insertions(+), 345 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..7b9c349343 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..e2895209a1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2b86..7ebb851e53 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -511,7 +511,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +529,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +999,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1208,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..0129db18c6 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc636b8..addf307cb6 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -439,37 +439,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
-/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
/*
* Check that the specified publications are present on the publisher.
*/
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..6f9e1269a0 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,24 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +930,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +986,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1034,25 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1064,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -995,6 +1087,20 @@ fetch_remote_table_info(char *nspname, char *relname,
continue;
}
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the generated column option
+ * is specified nor it appears in the column list, we will skip it.
+ */
+ if (remotegenlist[natt] && !has_pub_with_pubgencols &&
+ !bms_is_member(attnum, included_cols))
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot, 2, &isnull));
Assert(!isnull);
@@ -1015,7 +1121,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1143,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1229,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1244,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1300,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1371,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..d953a1afce 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,37 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
+ */
+static Bitmapset *
+prepare_nogen_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,13 +1073,18 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * Process potential column lists for the following cases: a. Any
+ * publication that is not FOR ALL TABLES. b. When the publication is
+ * FOR ALL TABLES and 'publish_generated_columns' is false. FOR ALL
+ * TABLES publication doesn't have user-defined column lists, so all
+ * columns will be replicated by default. However, if
+ * 'publish_generated_columns' is set to false, column lists must
+ * still be created to exclude any generated columns from being
+ * published.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1063,47 +1099,53 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ pub_rel_has_collist = !pub_no_list;
+ }
- pgoutput_ensure_entry_cxt(data, entry);
+ /* Build the column list bitmap in the per-entry context. */
+ if (pub_rel_has_collist || !pub->pubgencols)
+ {
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ pgoutput_ensure_entry_cxt(data, entry);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_nogen_columns_bms(data, entry, desc);
- if (att->attisdropped || att->attgenerated)
- continue;
+ /* Get the number of live attributes. */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- nliveatts++;
- }
+ if (att->attisdropped)
+ continue;
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ nliveatts++;
}
- ReleaseSysCache(cftuple);
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
+ }
}
+
+ if (HeapTupleIsValid(cftuple))
+ ReleaseSysCache(cftuple);
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91083..16cbef3693 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index b4efb127dc..41de3c93bb 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -3511,7 +3511,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..849b3a0804 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..158b444275 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..e419ca88d4 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,44 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..9724ba3f0d 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,42 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.41.0.windows.3
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-09 03:38 ` vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
5 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-09 03:38 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
>
> On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> >
> > Hi Shubham, here are my review comments for v36-0001.
> >
> > ======
> > 1. General - merge patches
> >
> > It is long past due when patches 0001 and 0002 should've been merged.
> > AFAIK the split was only because historically these parts had
> > different authors. But, keeping them separated is not helpful anymore.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 2.
> > Bitmapset *
> > -pub_collist_validate(Relation targetrel, List *columns)
> > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> >
> > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > so it should also be removed.
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 3.
> > /*
> > - * If the publication is FOR ALL TABLES then it is treated the same as
> > - * if there are no column lists (even if other publications have a
> > - * list).
> > + * To handle cases where the publish_generated_columns option isn't
> > + * specified for all tables in a publication, we must create a column
> > + * list that excludes generated columns. So, the publisher will not
> > + * replicate the generated columns.
> > */
> > - if (!pub->alltables)
> > + if (!(pub->alltables && pub->pubgencols))
> >
> > I still found that comment hard to understand. Does this mean to say
> > something like:
> >
> > ------
> > Process potential column lists for the following cases:
> >
> > a. Any publication that is not FOR ALL TABLES.
> >
> > b. When the publication is FOR ALL TABLES and
> > 'publish_generated_columns' is false.
> > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > so all columns will be replicated by default. However, if
> > 'publish_generated_columns' is set to false, column lists must still
> > be created to exclude any generated columns from being published
> > ------
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 4.
> > +SET client_min_messages = 'WARNING';
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> >
> > AFAIK you don't need to keep changing 'client_min_messages',
> > particularly now that you've removed the WARNING message that was
> > previously emitted.
> >
> > ~
> >
> > 5.
> > nit - minor comment changes.
> >
> > ======
> > Please refer to the attachment which implements any nits from above.
> >
>
> I have fixed all the given comments. Also, I have created a new 0003
> patch for the TAP-Tests related to the '011_generated.pl' file. I am
> planning to merge 0001 and 0003 patches once they will get fixed.
> The attached patches contain the required changes.
Few comments:
1) Since we are no longer throwing an error for generated columns, the
function header comments also need to be updated accordingly " Checks
for and raises an ERROR for any; unknown columns, system columns,
duplicate columns or generated columns."
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
-
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated
column \"%s\" in publication column list",
- colname));
-
2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
ALTER PUBLICATION ... SET (
postgres=# alter publication pub2 set (PUBLISH
PUBLISH PUBLISH_VIA_PARTITION_ROOT
3) I was able to compile without this include, may be this is not required:
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
4) You can include "\dRp+ pubname" after each of the create/alter
publication to verify the columns that will be published:
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
(publish_generated_columns=false);
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
(publish_generated_columns=true);
+
+-- Generated columns in column list, then set
'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- Remove generate columns from column list, when
'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-16 17:54 ` Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-17 10:29 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
0 siblings, 2 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-16 17:54 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
>
> On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> >
> > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > >
> > > Hi Shubham, here are my review comments for v36-0001.
> > >
> > > ======
> > > 1. General - merge patches
> > >
> > > It is long past due when patches 0001 and 0002 should've been merged.
> > > AFAIK the split was only because historically these parts had
> > > different authors. But, keeping them separated is not helpful anymore.
> > >
> > > ======
> > > src/backend/catalog/pg_publication.c
> > >
> > > 2.
> > > Bitmapset *
> > > -pub_collist_validate(Relation targetrel, List *columns)
> > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > >
> > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > so it should also be removed.
> > >
> > > ======
> > > src/backend/replication/pgoutput/pgoutput.c
> > >
> > > 3.
> > > /*
> > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > - * if there are no column lists (even if other publications have a
> > > - * list).
> > > + * To handle cases where the publish_generated_columns option isn't
> > > + * specified for all tables in a publication, we must create a column
> > > + * list that excludes generated columns. So, the publisher will not
> > > + * replicate the generated columns.
> > > */
> > > - if (!pub->alltables)
> > > + if (!(pub->alltables && pub->pubgencols))
> > >
> > > I still found that comment hard to understand. Does this mean to say
> > > something like:
> > >
> > > ------
> > > Process potential column lists for the following cases:
> > >
> > > a. Any publication that is not FOR ALL TABLES.
> > >
> > > b. When the publication is FOR ALL TABLES and
> > > 'publish_generated_columns' is false.
> > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > so all columns will be replicated by default. However, if
> > > 'publish_generated_columns' is set to false, column lists must still
> > > be created to exclude any generated columns from being published
> > > ------
> > >
> > > ======
> > > src/test/regress/sql/publication.sql
> > >
> > > 4.
> > > +SET client_min_messages = 'WARNING';
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > >
> > > AFAIK you don't need to keep changing 'client_min_messages',
> > > particularly now that you've removed the WARNING message that was
> > > previously emitted.
> > >
> > > ~
> > >
> > > 5.
> > > nit - minor comment changes.
> > >
> > > ======
> > > Please refer to the attachment which implements any nits from above.
> > >
> >
> > I have fixed all the given comments. Also, I have created a new 0003
> > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > planning to merge 0001 and 0003 patches once they will get fixed.
> > The attached patches contain the required changes.
>
> Few comments:
> 1) Since we are no longer throwing an error for generated columns, the
> function header comments also need to be updated accordingly " Checks
> for and raises an ERROR for any; unknown columns, system columns,
> duplicate columns or generated columns."
> - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> - ereport(ERROR,
> -
> errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> - errmsg("cannot use generated
> column \"%s\" in publication column list",
> - colname));
> -
>
> 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> ALTER PUBLICATION ... SET (
> postgres=# alter publication pub2 set (PUBLISH
> PUBLISH PUBLISH_VIA_PARTITION_ROOT
>
> 3) I was able to compile without this include, may be this is not required:
> --- a/src/backend/replication/logical/tablesync.c
> +++ b/src/backend/replication/logical/tablesync.c
> @@ -118,6 +118,7 @@
> #include "utils/builtins.h"
> #include "utils/lsyscache.h"
> #include "utils/memutils.h"
> +#include "utils/rel.h"
>
> 4) You can include "\dRp+ pubname" after each of the create/alter
> publication to verify the columns that will be published:
> +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> +-- different scenarios with/without generated columns in column lists.
> +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> +
> +-- Generated columns in column list, when 'publish_generated_columns'=false
> +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> (publish_generated_columns=false);
>
> +-- Generated columns in column list, when 'publish_generated_columns'=true
> +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> (publish_generated_columns=true);
> +
> +-- Generated columns in column list, then set
> 'publication_generate_columns'=false
> +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> +
> +-- Remove generate columns from column list, when
> 'publish_generated_columns'=false
> +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> +
> +-- Add generated columns in column list, when 'publish_generated_columns'=false
> +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
>
I have fixed all the given comments. The attached patches contain the
required changes.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v39-0002-DOCS-Generated-Column-Replication.patch (13.9K, ../../CAHv8RjLjb+98i5ZQUphivxdOZ3hSGLfq2SiWQetUvk8zGyAQwQ@mail.gmail.com/2-v39-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From 33aa1b0735bd5e87d0ef22faa200a79505806fc9 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 10 Oct 2024 11:19:22 +1100
Subject: [PATCH v39 2/3] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 4 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 4 +-
doc/src/sgml/ref/create_publication.sgml | 16 ++
4 files changed, 310 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..192180d658 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..7a8524e825 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,22 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
--
2.41.0.windows.3
[application/octet-stream] v39-0001-Enable-support-for-publish_generated_columns-opt.patch (98.0K, ../../CAHv8RjLjb+98i5ZQUphivxdOZ3hSGLfq2SiWQetUvk8zGyAQwQ@mail.gmail.com/3-v39-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 9d08c74b38ae29e3ba7173b1446527049c7dbbb7 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 8 Oct 2024 11:02:36 +0530
Subject: [PATCH v39 1/3] Enable support for 'publish_generated_columns'
option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
src/backend/catalog/pg_publication.c | 18 +-
src/backend/catalog/pg_subscription.c | 31 ++
src/backend/commands/publicationcmds.c | 33 +-
src/backend/commands/subscriptioncmds.c | 31 --
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 178 +++++--
src/backend/replication/pgoutput/pgoutput.c | 110 +++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_publication.h | 4 +
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 508 ++++++++++++--------
src/test/regress/sql/publication.sql | 45 +-
src/test/subscription/t/031_column_list.pl | 4 +-
20 files changed, 702 insertions(+), 337 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7e5e357fd9..f1da14fcb4 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -499,9 +499,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
/*
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
- * valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * valid to use for a publication. Checks for and raises an ERROR for
+ * any; unknown columns, system columns or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
@@ -511,7 +510,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +528,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +998,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1213,7 +1206,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..0129db18c6 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc636b8..addf307cb6 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -439,37 +439,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
}
-/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
/*
* Check that the specified publications are present on the publisher.
*/
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..621871396f 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -692,21 +692,59 @@ process_syncing_tables(XLogRecPtr current_lsn)
}
/*
- * Create list of columns for COPY based on logical relation mapping.
+ * Create a list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
make_copy_attnamelist(LogicalRepRelMapEntry *rel)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip adding
+ * this column to the column list for COPY.
+ */
+ if (remote_attnum >= 0)
+ localgenlist[remote_attnum] = true;
}
+ /*
+ * Construct a column list for COPY, excluding columns that are
+ * subscription table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +829,22 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,7 +887,8 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
@@ -873,8 +915,7 @@ fetch_remote_table_info(char *nspname, char *relname,
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
@@ -937,6 +978,43 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names.data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names.data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names.data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
pfree(pub_names.data);
}
@@ -948,20 +1026,22 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped"
" AND a.attrelid = %u"
- " ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
- lrel->remoteid);
+ " ORDER BY a.attnum", lrel->remoteid, lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1053,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -995,6 +1076,22 @@ fetch_remote_table_info(char *nspname, char *relname,
continue;
}
+ if (server_version >= 180000)
+ {
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the generated column
+ * option is specified nor it appears in the column list, we will
+ * skip it.
+ */
+ if (remotegenlist[natt] && !has_pub_with_pubgencols && !included_cols)
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot, 2, &isnull));
Assert(!isnull);
@@ -1015,7 +1112,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1134,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1220,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1235,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1291,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1362,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..d6b8d1b4f7 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,37 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
+ */
+static Bitmapset *
+prepare_nogen_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,13 +1073,20 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * Process potential column lists for the following cases:
+ *
+ * a. Any publication that is not FOR ALL TABLES.
+ *
+ * b. When the publication is FOR ALL TABLES and
+ * 'publish_generated_columns' is false. FOR ALL TABLES publication
+ * doesn't have user-defined column lists, so all columns will be
+ * replicated by default. However, if 'publish_generated_columns' is
+ * set to false, column lists must still be created to exclude any
+ * generated columns from being published.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1063,47 +1101,53 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ pub_rel_has_collist = !pub_no_list;
+ }
- pgoutput_ensure_entry_cxt(data, entry);
+ /* Build the column list bitmap in the per-entry context. */
+ if (pub_rel_has_collist || !pub->pubgencols)
+ {
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ pgoutput_ensure_entry_cxt(data, entry);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_nogen_columns_bms(data, entry, desc);
- if (att->attisdropped || att->attgenerated)
- continue;
+ /* Get the number of live attributes. */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- nliveatts++;
- }
+ if (att->attisdropped)
+ continue;
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ nliveatts++;
}
- ReleaseSysCache(cftuple);
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
+ }
}
+
+ if (HeapTupleIsValid(cftuple))
+ ReleaseSysCache(cftuple);
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91083..16cbef3693 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 1be0056af7..fad2277991 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2261,7 +2261,7 @@ match_previous_words(int pattern_id,
"CURRENT_SCHEMA");
/* ALTER PUBLICATION <name> SET ( */
else if (Matches("ALTER", "PUBLICATION", MatchAny, MatchAnyN, "SET", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..849b3a0804 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..158b444275 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..fc856d9a14 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,84 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | t
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a)
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..454a03bc3d 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,47 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.41.0.windows.3
[application/octet-stream] v39-0003-Tap-tests-for-generated-columns.patch (14.6K, ../../CAHv8RjLjb+98i5ZQUphivxdOZ3hSGLfq2SiWQetUvk8zGyAQwQ@mail.gmail.com/4-v39-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 53c5ab21f80b2eaf66569ac826bf7c953ef6b96f Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v39 3/3] Tap tests for generated columns
Add tests for the combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 354 +++++++++++++++++++++++
1 file changed, 354 insertions(+)
mode change 100644 => 100755 src/test/subscription/t/011_generated.pl
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
old mode 100644
new mode 100755
index 8b2e5f4708..d1f2718078
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,358 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for the combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, the combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false and copy_data=true.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true and copy_data=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+
+# =============================================================================
+# Misc test.
+#
+# A "normal -> generated" replication fails, reporting an error that the
+# subscriber side column is missing.
+#
+# In this test case we use DROP EXPRESSION to change the subscriber generated
+# column into a normal column, then verify replication works ok.
+# =============================================================================
+
+# Create publication and table with normal column 'b'
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_alter (a int, b int);
+ CREATE PUBLICATION regress_pub_alter FOR TABLE tab_alter;
+));
+
+# Create subscription and table with a generated column 'b'
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_alter (a int, b int GENERATED ALWAYS AS (a * 22) STORED);
+ CREATE SUBSCRIPTION regress_sub_alter CONNECTION '$publisher_connstr' PUBLICATION regress_pub_alter WITH (copy_data = false);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+# Change the generated column 'b' to be a normal column.
+$node_subscriber->safe_psql('postgres',
+ "ALTER TABLE tab_alter ALTER COLUMN b DROP EXPRESSION");
+
+# Insert data to verify replication.
+
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_alter VALUES (1,1), (2,2), (3,3)");
+
+# Verify that replication works, now that the subscriber column 'b' is normal
+$node_publisher->wait_for_catchup('regress_sub_alter');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_alter ORDER BY a");
+is( $result, qq(1|1
+2|2
+3|3), 'after drop generated column expression');
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub_alter");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub_alter");
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen(a, b, gen2), gen_to_nogen2, nogen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen VALUES (1, 1);
+ INSERT INTO gen_to_nogen2 VALUES (1, 1);
+ INSERT INTO nogen_to_gen2 VALUES (1, 1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=false and copy_data=true.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen2 ORDER BY c");
+is($result, qq(1|1||),
+ 'gen_to_nogen2 initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen2 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen VALUES (2), (3);
+ INSERT INTO gen_to_nogen2 VALUES (2), (3);
+ INSERT INTO nogen_to_gen2 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=false and copy_data=true.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen ORDER BY a");
+is( $result, qq(1|1||2
+2|||4
+3|||6),
+ 'gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen2 ORDER BY c");
+is( $result, qq(1|1||
+2|||
+3|||),
+ 'gen_to_nogen2 incremental replication, when publish_generated_columns=false'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen2 ORDER BY c");
+is( $result, qq(||2|
+||4|
+||6|),
+ 'nogen_to_gen2 incremental replication, when publish_generated_columns=false'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen3(a, b, gen2), gen_to_nogen4, nogen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen3 VALUES (1, 1);
+ INSERT INTO gen_to_nogen4 VALUES (1, 1);
+ INSERT INTO nogen_to_gen4 VALUES (1, 1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=true and copy_data=true.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen3 ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen4 ORDER BY c");
+is($result, qq(1|1|2|2),
+ 'gen_to_nogen4 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen4 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication.
+# Verify that column 'b' is replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen3 VALUES (2), (3);
+ INSERT INTO gen_to_nogen4 VALUES (2), (3);
+ INSERT INTO nogen_to_gen4 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=true and copy_data=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen3 ORDER BY a");
+is( $result, qq(1|1||2
+2|||4
+3|||6),
+ 'gen_to_nogen3 incremental replication, when publish_generated_columns=false'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen4 ORDER BY c");
+is( $result, qq(1|1|2|2
+2||4|4
+3||6|6),
+ 'gen_to_nogen4 incremental replication, when publish_generated_columns=false'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen4 ORDER BY c");
+is( $result, qq(||2|
+||4|
+||6|),
+ 'nogen_to_gen4 incremental replication, when publish_generated_columns=false'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.41.0.windows.3
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-17 07:27 ` vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-17 07:27 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, 16 Oct 2024 at 23:25, Shubham Khanna
<[email protected]> wrote:
>
> On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
> >
> > On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> > >
> > > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > > >
> > > > Hi Shubham, here are my review comments for v36-0001.
> > > >
> > > > ======
> > > > 1. General - merge patches
> > > >
> > > > It is long past due when patches 0001 and 0002 should've been merged.
> > > > AFAIK the split was only because historically these parts had
> > > > different authors. But, keeping them separated is not helpful anymore.
> > > >
> > > > ======
> > > > src/backend/catalog/pg_publication.c
> > > >
> > > > 2.
> > > > Bitmapset *
> > > > -pub_collist_validate(Relation targetrel, List *columns)
> > > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > > >
> > > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > > so it should also be removed.
> > > >
> > > > ======
> > > > src/backend/replication/pgoutput/pgoutput.c
> > > >
> > > > 3.
> > > > /*
> > > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > > - * if there are no column lists (even if other publications have a
> > > > - * list).
> > > > + * To handle cases where the publish_generated_columns option isn't
> > > > + * specified for all tables in a publication, we must create a column
> > > > + * list that excludes generated columns. So, the publisher will not
> > > > + * replicate the generated columns.
> > > > */
> > > > - if (!pub->alltables)
> > > > + if (!(pub->alltables && pub->pubgencols))
> > > >
> > > > I still found that comment hard to understand. Does this mean to say
> > > > something like:
> > > >
> > > > ------
> > > > Process potential column lists for the following cases:
> > > >
> > > > a. Any publication that is not FOR ALL TABLES.
> > > >
> > > > b. When the publication is FOR ALL TABLES and
> > > > 'publish_generated_columns' is false.
> > > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > > so all columns will be replicated by default. However, if
> > > > 'publish_generated_columns' is set to false, column lists must still
> > > > be created to exclude any generated columns from being published
> > > > ------
> > > >
> > > > ======
> > > > src/test/regress/sql/publication.sql
> > > >
> > > > 4.
> > > > +SET client_min_messages = 'WARNING';
> > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > >
> > > > AFAIK you don't need to keep changing 'client_min_messages',
> > > > particularly now that you've removed the WARNING message that was
> > > > previously emitted.
> > > >
> > > > ~
> > > >
> > > > 5.
> > > > nit - minor comment changes.
> > > >
> > > > ======
> > > > Please refer to the attachment which implements any nits from above.
> > > >
> > >
> > > I have fixed all the given comments. Also, I have created a new 0003
> > > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > > planning to merge 0001 and 0003 patches once they will get fixed.
> > > The attached patches contain the required changes.
> >
> > Few comments:
> > 1) Since we are no longer throwing an error for generated columns, the
> > function header comments also need to be updated accordingly " Checks
> > for and raises an ERROR for any; unknown columns, system columns,
> > duplicate columns or generated columns."
> > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > - ereport(ERROR,
> > -
> > errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > - errmsg("cannot use generated
> > column \"%s\" in publication column list",
> > - colname));
> > -
> >
> > 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> > ALTER PUBLICATION ... SET (
> > postgres=# alter publication pub2 set (PUBLISH
> > PUBLISH PUBLISH_VIA_PARTITION_ROOT
> >
> > 3) I was able to compile without this include, may be this is not required:
> > --- a/src/backend/replication/logical/tablesync.c
> > +++ b/src/backend/replication/logical/tablesync.c
> > @@ -118,6 +118,7 @@
> > #include "utils/builtins.h"
> > #include "utils/lsyscache.h"
> > #include "utils/memutils.h"
> > +#include "utils/rel.h"
> >
> > 4) You can include "\dRp+ pubname" after each of the create/alter
> > publication to verify the columns that will be published:
> > +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> > +-- different scenarios with/without generated columns in column lists.
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > +
> > +-- Generated columns in column list, when 'publish_generated_columns'=false
> > +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> > (publish_generated_columns=false);
> >
> > +-- Generated columns in column list, when 'publish_generated_columns'=true
> > +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> > (publish_generated_columns=true);
> > +
> > +-- Generated columns in column list, then set
> > 'publication_generate_columns'=false
> > +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> > +
> > +-- Remove generate columns from column list, when
> > 'publish_generated_columns'=false
> > +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> > +
> > +-- Add generated columns in column list, when 'publish_generated_columns'=false
> > +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
> >
>
> I have fixed all the given comments. The attached patches contain the
> required changes.
Few comments:
1) This change is not required:
diff --git a/src/backend/catalog/pg_subscription.c
b/src/backend/catalog/pg_subscription.c
index 9efc9159f2..fcfbf86c0b 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest,
quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
It can be moved to subscriptioncmds.c file as earlier.
2) This line change is not required:
* Process and validate the 'columns' list and ensure the
columns are all
- * valid to use for a publication. Checks for and raises
an ERROR for
- * any; unknown columns, system columns, duplicate
columns or generated
- * columns.
+ * valid to use for a publication. Checks for and raises
an ERROR for
3) Can we store this information in LogicalRepRelation instead of
having a local variable as column information is being stored, that
way remotegenlist and remotegenlist_res can be removed and code will
be more simpler:
+ if (server_version >= 180000)
+ {
+ remotegenlist[natt] =
DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the
generated column
+ * option is specified nor it appears in the
column list, we will
+ * skip it.
+ */
+ if (remotegenlist[natt] &&
!has_pub_with_pubgencols && !included_cols)
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot,
2, &isnull));
Assert(!isnull);
@@ -1015,7 +1112,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-18 12:12 ` Shubham Khanna <[email protected]>
2024-10-21 03:37 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-21 05:21 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-21 11:19 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 01:02 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-22 07:28 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-23 05:00 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 6 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-18 12:12 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, Oct 17, 2024 at 12:58 PM vignesh C <[email protected]> wrote:
>
> On Wed, 16 Oct 2024 at 23:25, Shubham Khanna
> <[email protected]> wrote:
> >
> > On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
> > >
> > > On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> > > >
> > > > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > > > >
> > > > > Hi Shubham, here are my review comments for v36-0001.
> > > > >
> > > > > ======
> > > > > 1. General - merge patches
> > > > >
> > > > > It is long past due when patches 0001 and 0002 should've been merged.
> > > > > AFAIK the split was only because historically these parts had
> > > > > different authors. But, keeping them separated is not helpful anymore.
> > > > >
> > > > > ======
> > > > > src/backend/catalog/pg_publication.c
> > > > >
> > > > > 2.
> > > > > Bitmapset *
> > > > > -pub_collist_validate(Relation targetrel, List *columns)
> > > > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > > > >
> > > > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > > > so it should also be removed.
> > > > >
> > > > > ======
> > > > > src/backend/replication/pgoutput/pgoutput.c
> > > > >
> > > > > 3.
> > > > > /*
> > > > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > > > - * if there are no column lists (even if other publications have a
> > > > > - * list).
> > > > > + * To handle cases where the publish_generated_columns option isn't
> > > > > + * specified for all tables in a publication, we must create a column
> > > > > + * list that excludes generated columns. So, the publisher will not
> > > > > + * replicate the generated columns.
> > > > > */
> > > > > - if (!pub->alltables)
> > > > > + if (!(pub->alltables && pub->pubgencols))
> > > > >
> > > > > I still found that comment hard to understand. Does this mean to say
> > > > > something like:
> > > > >
> > > > > ------
> > > > > Process potential column lists for the following cases:
> > > > >
> > > > > a. Any publication that is not FOR ALL TABLES.
> > > > >
> > > > > b. When the publication is FOR ALL TABLES and
> > > > > 'publish_generated_columns' is false.
> > > > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > > > so all columns will be replicated by default. However, if
> > > > > 'publish_generated_columns' is set to false, column lists must still
> > > > > be created to exclude any generated columns from being published
> > > > > ------
> > > > >
> > > > > ======
> > > > > src/test/regress/sql/publication.sql
> > > > >
> > > > > 4.
> > > > > +SET client_min_messages = 'WARNING';
> > > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > > >
> > > > > AFAIK you don't need to keep changing 'client_min_messages',
> > > > > particularly now that you've removed the WARNING message that was
> > > > > previously emitted.
> > > > >
> > > > > ~
> > > > >
> > > > > 5.
> > > > > nit - minor comment changes.
> > > > >
> > > > > ======
> > > > > Please refer to the attachment which implements any nits from above.
> > > > >
> > > >
> > > > I have fixed all the given comments. Also, I have created a new 0003
> > > > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > > > planning to merge 0001 and 0003 patches once they will get fixed.
> > > > The attached patches contain the required changes.
> > >
> > > Few comments:
> > > 1) Since we are no longer throwing an error for generated columns, the
> > > function header comments also need to be updated accordingly " Checks
> > > for and raises an ERROR for any; unknown columns, system columns,
> > > duplicate columns or generated columns."
> > > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > > - ereport(ERROR,
> > > -
> > > errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > > - errmsg("cannot use generated
> > > column \"%s\" in publication column list",
> > > - colname));
> > > -
> > >
> > > 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> > > ALTER PUBLICATION ... SET (
> > > postgres=# alter publication pub2 set (PUBLISH
> > > PUBLISH PUBLISH_VIA_PARTITION_ROOT
> > >
> > > 3) I was able to compile without this include, may be this is not required:
> > > --- a/src/backend/replication/logical/tablesync.c
> > > +++ b/src/backend/replication/logical/tablesync.c
> > > @@ -118,6 +118,7 @@
> > > #include "utils/builtins.h"
> > > #include "utils/lsyscache.h"
> > > #include "utils/memutils.h"
> > > +#include "utils/rel.h"
> > >
> > > 4) You can include "\dRp+ pubname" after each of the create/alter
> > > publication to verify the columns that will be published:
> > > +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> > > +-- different scenarios with/without generated columns in column lists.
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > +
> > > +-- Generated columns in column list, when 'publish_generated_columns'=false
> > > +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> > > (publish_generated_columns=false);
> > >
> > > +-- Generated columns in column list, when 'publish_generated_columns'=true
> > > +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> > > (publish_generated_columns=true);
> > > +
> > > +-- Generated columns in column list, then set
> > > 'publication_generate_columns'=false
> > > +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> > > +
> > > +-- Remove generate columns from column list, when
> > > 'publish_generated_columns'=false
> > > +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> > > +
> > > +-- Add generated columns in column list, when 'publish_generated_columns'=false
> > > +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
> > >
> >
> > I have fixed all the given comments. The attached patches contain the
> > required changes.
>
> Few comments:
> 1) This change is not required:
> diff --git a/src/backend/catalog/pg_subscription.c
> b/src/backend/catalog/pg_subscription.c
> index 9efc9159f2..fcfbf86c0b 100644
> --- a/src/backend/catalog/pg_subscription.c
> +++ b/src/backend/catalog/pg_subscription.c
> @@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
>
> return res;
> }
> +
> +/*
> + * Add publication names from the list to a string.
> + */
> +void
> +get_publications_str(List *publications, StringInfo dest, bool quote_literal)
> +{
> + ListCell *lc;
> + bool first = true;
> +
> + Assert(publications != NIL);
> +
> + foreach(lc, publications)
> + {
> + char *pubname = strVal(lfirst(lc));
> +
> + if (first)
> + first = false;
> + else
> + appendStringInfoString(dest, ", ");
> +
> + if (quote_literal)
> + appendStringInfoString(dest,
> quote_literal_cstr(pubname));
> + else
> + {
> + appendStringInfoChar(dest, '"');
> + appendStringInfoString(dest, pubname);
> + appendStringInfoChar(dest, '"');
> + }
> + }
> +}
>
> It can be moved to subscriptioncmds.c file as earlier.
>
> 2) This line change is not required:
> * Process and validate the 'columns' list and ensure the
> columns are all
> - * valid to use for a publication. Checks for and raises
> an ERROR for
> - * any; unknown columns, system columns, duplicate
> columns or generated
> - * columns.
> + * valid to use for a publication. Checks for and raises
> an ERROR for
>
> 3) Can we store this information in LogicalRepRelation instead of
> having a local variable as column information is being stored, that
> way remotegenlist and remotegenlist_res can be removed and code will
> be more simpler:
> + if (server_version >= 180000)
> + {
> + remotegenlist[natt] =
> DatumGetBool(slot_getattr(slot, 5, &isnull));
> +
> + /*
> + * If the column is generated and neither the
> generated column
> + * option is specified nor it appears in the
> column list, we will
> + * skip it.
> + */
> + if (remotegenlist[natt] &&
> !has_pub_with_pubgencols && !included_cols)
> + {
> + ExecClearTuple(slot);
> + continue;
> + }
> + }
> +
> rel_colname = TextDatumGetCString(slot_getattr(slot,
> 2, &isnull));
> Assert(!isnull);
>
> @@ -1015,7 +1112,7 @@ fetch_remote_table_info(char *nspname, char *relname,
> ExecDropSingleTupleTableSlot(slot);
>
> lrel->natts = natt;
> -
> + *remotegenlist_res = remotegenlist;
I have fixed all the given comments. The attached v40-0001 patch
contains the required changes.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v40-0003-Tap-tests-for-generated-columns.patch (12.8K, ../../CAHv8RjLviXAWtB3Kcn1A1jPpqORpkNay1y2U+55K64sqwCdrGw@mail.gmail.com/2-v40-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 83542ba0d65b8022ed6f8ffab7d2a75508ec2288 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v40 3/3] Tap tests for generated columns
Add tests for the combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 329 +++++++++++++++++++++++
1 file changed, 329 insertions(+)
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index 8b2e5f4708..ff44c87af0 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,333 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for the combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, the combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+$node_subscriber->safe_psql('test_pgc_true', "DROP table tab_gen_to_nogen");
+$node_subscriber->safe_psql('postgres', "DROP DATABASE test_pgc_true");
+
+# =============================================================================
+# Misc test.
+#
+# A "normal -> generated" replication.
+#
+# In this test case we use DROP EXPRESSION to change the subscriber generated
+# column into a normal column, then verify replication works ok.
+# =============================================================================
+
+# Create publication and table with normal column 'b'
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_alter (a int, b int);
+ CREATE PUBLICATION regress_pub_alter FOR TABLE tab_alter;
+));
+
+# Create subscription and table with a generated column 'b'
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_alter (a int, b int GENERATED ALWAYS AS (a * 22) STORED);
+ CREATE SUBSCRIPTION regress_sub_alter CONNECTION '$publisher_connstr' PUBLICATION regress_pub_alter WITH (copy_data = false);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+# Change the generated column 'b' to be a normal column.
+$node_subscriber->safe_psql('postgres',
+ "ALTER TABLE tab_alter ALTER COLUMN b DROP EXPRESSION");
+
+# Insert data to verify replication.
+
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_alter VALUES (1,1), (2,2), (3,3)");
+
+# Verify that replication works, now that the subscriber column 'b' is normal
+$node_publisher->wait_for_catchup('regress_sub_alter');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_alter ORDER BY a");
+is( $result, qq(1|1
+2|2
+3|3), 'after drop generated column expression');
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub_alter");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub_alter");
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen, tab_gen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen2 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=false.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|),
+ 'tab_gen_to_gen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen2 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|
+2|
+3|),
+ 'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen2 incremental replication, when publish_generated_columns=false'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen3, tab_gen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen4 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2),
+ 'tab_gen_to_gen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication.
+# Verify that column 'b' is replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen4 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2
+2|4
+3|6),
+ 'tab_gen_to_gen3 incremental replication, when publish_generated_columns=true'
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen4 incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.34.1
[application/octet-stream] v40-0002-DOCS-Generated-Column-Replication.patch (13.9K, ../../CAHv8RjLviXAWtB3Kcn1A1jPpqORpkNay1y2U+55K64sqwCdrGw@mail.gmail.com/3-v40-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From f2af9172f7bcf65f6b86ce7a94459e46d971c969 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 10 Oct 2024 11:19:22 +1100
Subject: [PATCH v40 2/3] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 4 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 4 +-
doc/src/sgml/ref/create_publication.sgml | 16 ++
4 files changed, 310 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..192180d658 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..7a8524e825 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,22 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
--
2.34.1
[application/octet-stream] v40-0001-Enable-support-for-publish_generated_columns-opt.patch (100.4K, ../../CAHv8RjLviXAWtB3Kcn1A1jPpqORpkNay1y2U+55K64sqwCdrGw@mail.gmail.com/4-v40-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 5f4dde093cf5d3bebca42568ed6dfc91824a76a4 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 8 Oct 2024 11:02:36 +0530
Subject: [PATCH v40] Enable support for 'publish_generated_columns' option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
src/backend/catalog/pg_publication.c | 13 +-
src/backend/commands/publicationcmds.c | 79 ++-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 173 +++++--
src/backend/replication/pgoutput/pgoutput.c | 110 +++--
src/backend/utils/cache/relcache.c | 10 +-
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_publication.h | 4 +
src/include/catalog/pg_subscription.h | 1 +
src/include/commands/publicationcmds.h | 3 +-
src/include/replication/logicalproto.h | 2 +
src/include/replication/logicalrelation.h | 3 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 508 ++++++++++++--------
src/test/regress/sql/publication.sql | 45 +-
src/test/subscription/t/031_column_list.pl | 4 +-
src/test/subscription/t/100_bugs.pl | 4 +-
22 files changed, 712 insertions(+), 317 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7e5e357fd9..e6e5506f58 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -511,7 +511,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +529,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +999,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1213,7 +1207,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..d564cb4e8e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -332,7 +343,7 @@ pub_rf_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
*/
bool
pub_collist_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
- bool pubviaroot)
+ bool pubviaroot, bool pubgencols)
{
HeapTuple tuple;
Oid relid = RelationGetRelid(relation);
@@ -368,18 +379,50 @@ pub_collist_contains_invalid_column(Oid pubid, Relation relation, List *ancestor
Anum_pg_publication_rel_prattrs,
&isnull);
- if (!isnull)
+ if (!isnull || !pubgencols)
{
int x;
Bitmapset *idattrs;
Bitmapset *columns = NULL;
- /* With REPLICA IDENTITY FULL, no column list is allowed. */
- if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
- result = true;
+ if (!isnull)
+ {
+ /* With REPLICA IDENTITY FULL, no column list is allowed. */
+ if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
+ result = true;
+
+ /* Transform the column list datum to a bitmapset. */
+ columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ }
+ else
+ {
+ TupleDesc desc = RelationGetDescr(relation);
+ int nliveatts = 0;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped)
+ continue;
+
+ nliveatts++;
+
+ if (att->attgenerated)
+ continue;
+
+ columns = bms_add_member(columns, i + 1);
+ }
- /* Transform the column list datum to a bitmapset. */
- columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ /* Return if all columns of the table will be replicated */
+ if (bms_num_members(columns) == nliveatts)
+ {
+ bms_free(columns);
+ ReleaseSysCache(tuple);
+ return false;
+ }
+ }
/* Remember columns that are part of the REPLICA IDENTITY */
idattrs = RelationGetIndexAttrBitmap(relation,
@@ -737,6 +780,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +821,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +840,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +927,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +938,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1050,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..6b085e555c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..2f55fc5a35 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -692,21 +692,59 @@ process_syncing_tables(XLogRecPtr current_lsn)
}
/*
- * Create list of columns for COPY based on logical relation mapping.
+ * Create a list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
make_copy_attnamelist(LogicalRepRelMapEntry *rel)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip adding
+ * this column to the column list for COPY.
+ */
+ if (remote_attnum >= 0)
+ localgenlist[remote_attnum] = true;
}
+ /*
+ * Construct a column list for COPY, excluding columns that are
+ * subscription table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +829,20 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
- LogicalRepRelation *lrel, List **qual)
+fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool has_pub_with_pubgencols = false;
int natt;
ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,7 +885,8 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
@@ -873,8 +913,7 @@ fetch_remote_table_info(char *nspname, char *relname,
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
@@ -937,6 +976,43 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names.data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names.data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names.data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
pfree(pub_names.data);
}
@@ -948,20 +1024,22 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped"
" AND a.attrelid = %u"
- " ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
- lrel->remoteid);
+ " ORDER BY a.attnum", lrel->remoteid, lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1051,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ lrel->remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -995,6 +1074,22 @@ fetch_remote_table_info(char *nspname, char *relname,
continue;
}
+ if (server_version >= 180000)
+ {
+ lrel->remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the generated column
+ * option is specified nor it appears in the column list, we will
+ * skip it.
+ */
+ if (lrel->remotegenlist[natt] && !has_pub_with_pubgencols && !included_cols)
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot, 2, &isnull));
Assert(!isnull);
@@ -1015,7 +1110,6 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
walrcv_clear_result(res);
/*
@@ -1037,7 +1131,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,6 +1217,7 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
@@ -1135,11 +1230,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (lrel.remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1286,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1357,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..d6b8d1b4f7 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1008,6 +1008,37 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
+ */
+static Bitmapset *
+prepare_nogen_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
/*
* Initialize the column list.
*/
@@ -1042,13 +1073,20 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * Process potential column lists for the following cases:
+ *
+ * a. Any publication that is not FOR ALL TABLES.
+ *
+ * b. When the publication is FOR ALL TABLES and
+ * 'publish_generated_columns' is false. FOR ALL TABLES publication
+ * doesn't have user-defined column lists, so all columns will be
+ * replicated by default. However, if 'publish_generated_columns' is
+ * set to false, column lists must still be created to exclude any
+ * generated columns from being published.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1063,47 +1101,53 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ pub_rel_has_collist = !pub_no_list;
+ }
- pgoutput_ensure_entry_cxt(data, entry);
+ /* Build the column list bitmap in the per-entry context. */
+ if (pub_rel_has_collist || !pub->pubgencols)
+ {
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ pgoutput_ensure_entry_cxt(data, entry);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_nogen_columns_bms(data, entry, desc);
- if (att->attisdropped || att->attgenerated)
- continue;
+ /* Get the number of live attributes. */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- nliveatts++;
- }
+ if (att->attisdropped)
+ continue;
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ nliveatts++;
}
- ReleaseSysCache(cftuple);
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
+ }
}
+
+ if (HeapTupleIsValid(cftuple))
+ ReleaseSysCache(cftuple);
}
if (first)
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c326f687eb..80e11a37aa 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5826,13 +5826,15 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
/*
* Check if all columns are part of the REPLICA IDENTITY index or not.
*
- * If the publication is FOR ALL TABLES then it means the table has no
- * column list and we can skip the validation.
+ * If the publication is FOR ALL TABLES and publication includes
+ * generated columns then it means that all the table will replicate
+ * all columns and we can skip the validation.
*/
- if (!pubform->puballtables &&
+ if (!(pubform->puballtables && pubform->pubgencols) &&
(pubform->pubupdate || pubform->pubdelete) &&
pub_collist_contains_invalid_column(pubid, relation, ancestors,
- pubform->pubviaroot))
+ pubform->pubviaroot,
+ pubform->pubgencols))
{
if (pubform->pubupdate)
pubdesc->cols_valid_for_update = false;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e718..f9b38edd7e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6233,7 +6233,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6268,7 +6268,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6357,6 +6360,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6373,6 +6377,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6386,6 +6391,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6437,6 +6445,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6451,6 +6461,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6461,6 +6473,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 1be0056af7..fad2277991 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2261,7 +2261,7 @@ match_previous_words(int pattern_id,
"CURRENT_SCHEMA");
/* ALTER PUBLICATION <name> SET ( */
else if (Matches("ALTER", "PUBLICATION", MatchAny, MatchAnyN, "SET", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..849b3a0804 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..6657186317 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
diff --git a/src/include/commands/publicationcmds.h b/src/include/commands/publicationcmds.h
index 5487c571f6..ca796d6c02 100644
--- a/src/include/commands/publicationcmds.h
+++ b/src/include/commands/publicationcmds.h
@@ -34,6 +34,7 @@ extern void InvalidatePublicationRels(List *relids);
extern bool pub_rf_contains_invalid_column(Oid pubid, Relation relation,
List *ancestors, bool pubviaroot);
extern bool pub_collist_contains_invalid_column(Oid pubid, Relation relation,
- List *ancestors, bool pubviaroot);
+ List *ancestors, bool pubviaroot,
+ bool pubgencols);
#endif /* PUBLICATIONCMDS_H */
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index c409638a2e..e0b3a4bedb 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -113,6 +113,8 @@ typedef struct LogicalRepRelation
char replident; /* replica identity */
char relkind; /* remote relation kind */
Bitmapset *attkeys; /* Bitmap of key columns */
+ bool *remotegenlist; /* Array to store whether each column is
+ * generated */
} LogicalRepRelation;
/* Type mapping info */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..fc856d9a14 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,84 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | t
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a)
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..454a03bc3d 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,47 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index cb36ca7b16..64b902db73 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -391,7 +391,7 @@ $node_publisher->safe_psql(
ALTER TABLE dropped_cols REPLICA IDENTITY FULL;
CREATE TABLE generated_cols (a int, b_gen int GENERATED ALWAYS AS (5 * a) STORED, c int);
ALTER TABLE generated_cols REPLICA IDENTITY FULL;
- CREATE PUBLICATION pub_dropped_cols FOR TABLE dropped_cols, generated_cols;
+ CREATE PUBLICATION pub_dropped_cols FOR TABLE dropped_cols, generated_cols with (publish_generated_columns = true);
-- some initial data
INSERT INTO dropped_cols VALUES (1, 1, 1);
INSERT INTO generated_cols (a, c) VALUES (1, 1);
@@ -400,7 +400,7 @@ $node_publisher->safe_psql(
$node_subscriber->safe_psql(
'postgres', qq(
CREATE TABLE dropped_cols (a int, b_drop int, c int);
- CREATE TABLE generated_cols (a int, b_gen int GENERATED ALWAYS AS (5 * a) STORED, c int);
+ CREATE TABLE generated_cols (a int, b_gen int, c int);
));
$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
--
2.34.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-21 03:37 ` vignesh C <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: vignesh C @ 2024-10-21 03:37 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, 18 Oct 2024 at 17:42, Shubham Khanna
<[email protected]> wrote:
>
>
> I have fixed all the given comments. The attached v40-0001 patch
> contains the required changes.
1) The recent patch removed the function header comment where
generated column is specified, that change is required:
@@ -511,7 +511,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +529,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system
column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
-
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated
column \"%s\" in publication column list",
- colname));
-
2) This change is no more required as get_publications_str changes are
removed now:
diff --git a/src/include/catalog/pg_subscription.h
b/src/include/catalog/pg_subscription.h
index 0aa14ec4a2..6657186317 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-21 05:21 ` vignesh C <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: vignesh C @ 2024-10-21 05:21 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, 18 Oct 2024 at 17:42, Shubham Khanna
<[email protected]> wrote:
>
> On Thu, Oct 17, 2024 at 12:58 PM vignesh C <[email protected]> wrote:
> >
> > On Wed, 16 Oct 2024 at 23:25, Shubham Khanna
> > <[email protected]> wrote:
> > >
> > > On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
> > > >
> > > > On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> > > > >
> > > > > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > > > > >
> > > > > > Hi Shubham, here are my review comments for v36-0001.
> > > > > >
> > > > > > ======
> > > > > > 1. General - merge patches
> > > > > >
> > > > > > It is long past due when patches 0001 and 0002 should've been merged.
> > > > > > AFAIK the split was only because historically these parts had
> > > > > > different authors. But, keeping them separated is not helpful anymore.
> > > > > >
> > > > > > ======
> > > > > > src/backend/catalog/pg_publication.c
> > > > > >
> > > > > > 2.
> > > > > > Bitmapset *
> > > > > > -pub_collist_validate(Relation targetrel, List *columns)
> > > > > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > > > > >
> > > > > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > > > > so it should also be removed.
> > > > > >
> > > > > > ======
> > > > > > src/backend/replication/pgoutput/pgoutput.c
> > > > > >
> > > > > > 3.
> > > > > > /*
> > > > > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > > > > - * if there are no column lists (even if other publications have a
> > > > > > - * list).
> > > > > > + * To handle cases where the publish_generated_columns option isn't
> > > > > > + * specified for all tables in a publication, we must create a column
> > > > > > + * list that excludes generated columns. So, the publisher will not
> > > > > > + * replicate the generated columns.
> > > > > > */
> > > > > > - if (!pub->alltables)
> > > > > > + if (!(pub->alltables && pub->pubgencols))
> > > > > >
> > > > > > I still found that comment hard to understand. Does this mean to say
> > > > > > something like:
> > > > > >
> > > > > > ------
> > > > > > Process potential column lists for the following cases:
> > > > > >
> > > > > > a. Any publication that is not FOR ALL TABLES.
> > > > > >
> > > > > > b. When the publication is FOR ALL TABLES and
> > > > > > 'publish_generated_columns' is false.
> > > > > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > > > > so all columns will be replicated by default. However, if
> > > > > > 'publish_generated_columns' is set to false, column lists must still
> > > > > > be created to exclude any generated columns from being published
> > > > > > ------
> > > > > >
> > > > > > ======
> > > > > > src/test/regress/sql/publication.sql
> > > > > >
> > > > > > 4.
> > > > > > +SET client_min_messages = 'WARNING';
> > > > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > > > >
> > > > > > AFAIK you don't need to keep changing 'client_min_messages',
> > > > > > particularly now that you've removed the WARNING message that was
> > > > > > previously emitted.
> > > > > >
> > > > > > ~
> > > > > >
> > > > > > 5.
> > > > > > nit - minor comment changes.
> > > > > >
> > > > > > ======
> > > > > > Please refer to the attachment which implements any nits from above.
> > > > > >
> > > > >
> > > > > I have fixed all the given comments. Also, I have created a new 0003
> > > > > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > > > > planning to merge 0001 and 0003 patches once they will get fixed.
> > > > > The attached patches contain the required changes.
> > > >
> > > > Few comments:
> > > > 1) Since we are no longer throwing an error for generated columns, the
> > > > function header comments also need to be updated accordingly " Checks
> > > > for and raises an ERROR for any; unknown columns, system columns,
> > > > duplicate columns or generated columns."
> > > > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > > > - ereport(ERROR,
> > > > -
> > > > errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > > > - errmsg("cannot use generated
> > > > column \"%s\" in publication column list",
> > > > - colname));
> > > > -
> > > >
> > > > 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> > > > ALTER PUBLICATION ... SET (
> > > > postgres=# alter publication pub2 set (PUBLISH
> > > > PUBLISH PUBLISH_VIA_PARTITION_ROOT
> > > >
> > > > 3) I was able to compile without this include, may be this is not required:
> > > > --- a/src/backend/replication/logical/tablesync.c
> > > > +++ b/src/backend/replication/logical/tablesync.c
> > > > @@ -118,6 +118,7 @@
> > > > #include "utils/builtins.h"
> > > > #include "utils/lsyscache.h"
> > > > #include "utils/memutils.h"
> > > > +#include "utils/rel.h"
> > > >
> > > > 4) You can include "\dRp+ pubname" after each of the create/alter
> > > > publication to verify the columns that will be published:
> > > > +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> > > > +-- different scenarios with/without generated columns in column lists.
> > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > > +
> > > > +-- Generated columns in column list, when 'publish_generated_columns'=false
> > > > +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> > > > (publish_generated_columns=false);
> > > >
> > > > +-- Generated columns in column list, when 'publish_generated_columns'=true
> > > > +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> > > > (publish_generated_columns=true);
> > > > +
> > > > +-- Generated columns in column list, then set
> > > > 'publication_generate_columns'=false
> > > > +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> > > > +
> > > > +-- Remove generate columns from column list, when
> > > > 'publish_generated_columns'=false
> > > > +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> > > > +
> > > > +-- Add generated columns in column list, when 'publish_generated_columns'=false
> > > > +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
> > > >
> > >
> > > I have fixed all the given comments. The attached patches contain the
> > > required changes.
> >
> > Few comments:
> > 1) This change is not required:
> > diff --git a/src/backend/catalog/pg_subscription.c
> > b/src/backend/catalog/pg_subscription.c
> > index 9efc9159f2..fcfbf86c0b 100644
> > --- a/src/backend/catalog/pg_subscription.c
> > +++ b/src/backend/catalog/pg_subscription.c
> > @@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
> >
> > return res;
> > }
> > +
> > +/*
> > + * Add publication names from the list to a string.
> > + */
> > +void
> > +get_publications_str(List *publications, StringInfo dest, bool quote_literal)
> > +{
> > + ListCell *lc;
> > + bool first = true;
> > +
> > + Assert(publications != NIL);
> > +
> > + foreach(lc, publications)
> > + {
> > + char *pubname = strVal(lfirst(lc));
> > +
> > + if (first)
> > + first = false;
> > + else
> > + appendStringInfoString(dest, ", ");
> > +
> > + if (quote_literal)
> > + appendStringInfoString(dest,
> > quote_literal_cstr(pubname));
> > + else
> > + {
> > + appendStringInfoChar(dest, '"');
> > + appendStringInfoString(dest, pubname);
> > + appendStringInfoChar(dest, '"');
> > + }
> > + }
> > +}
> >
> > It can be moved to subscriptioncmds.c file as earlier.
> >
> > 2) This line change is not required:
> > * Process and validate the 'columns' list and ensure the
> > columns are all
> > - * valid to use for a publication. Checks for and raises
> > an ERROR for
> > - * any; unknown columns, system columns, duplicate
> > columns or generated
> > - * columns.
> > + * valid to use for a publication. Checks for and raises
> > an ERROR for
> >
> > 3) Can we store this information in LogicalRepRelation instead of
> > having a local variable as column information is being stored, that
> > way remotegenlist and remotegenlist_res can be removed and code will
> > be more simpler:
> > + if (server_version >= 180000)
> > + {
> > + remotegenlist[natt] =
> > DatumGetBool(slot_getattr(slot, 5, &isnull));
> > +
> > + /*
> > + * If the column is generated and neither the
> > generated column
> > + * option is specified nor it appears in the
> > column list, we will
> > + * skip it.
> > + */
> > + if (remotegenlist[natt] &&
> > !has_pub_with_pubgencols && !included_cols)
> > + {
> > + ExecClearTuple(slot);
> > + continue;
> > + }
> > + }
> > +
> > rel_colname = TextDatumGetCString(slot_getattr(slot,
> > 2, &isnull));
> > Assert(!isnull);
> >
> > @@ -1015,7 +1112,7 @@ fetch_remote_table_info(char *nspname, char *relname,
> > ExecDropSingleTupleTableSlot(slot);
> >
> > lrel->natts = natt;
> > -
> > + *remotegenlist_res = remotegenlist;
>
> I have fixed all the given comments. The attached v40-0001 patch
> contains the required changes.
Few comments:
1) Add a test case to ensure that an error is properly raised for the
issue reported by Swada-san in [1]:
create table t (a int not null, b int generated always as (a + 1)
stored not null);
create unique index t_idx on t (b);
alter table t replica identity using index t_idx;
insert into t values (1);
update t set a = 100 where a = 1;
2) The existing comments only reference the column list, so we should
revise them to include an important point about generated columns. If
generated columns are set to false, it may result in some columns not
being replicated:
+ if (!isnull)
+ {
+ /* With REPLICA IDENTITY FULL, no column list
is allowed. */
+ if (relation->rd_rel->relreplident ==
REPLICA_IDENTITY_FULL)
+ result = true;
+
+ /* Transform the column list datum to a bitmapset. */
+ columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ }
+ else
+ {
+ TupleDesc desc = RelationGetDescr(relation);
+ int nliveatts = 0;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or
generated */
+ if (att->attisdropped)
+ continue;
+
+ nliveatts++;
+
+ if (att->attgenerated)
+ continue;
+
+ columns = bms_add_member(columns, i + 1);
+ }
3) Now that we are sending generated columns from the publisher, the
comment in the tuples_equal function is no longer accurate. We need to
update the comment to reflect the new behavior of the function.
Specifically, it should clarify how generated columns are considered
in the equality check:
/*
* Compare the tuples in the slots by checking if they have equal values.
*/
static bool
tuples_equal(TupleTableSlot *slot1, TupleTableSlot *slot2,
TypeCacheEntry **eq)
{
....
/*
* Ignore dropped and generated columns as the publisher doesn't send
* those
*/
if (att->attisdropped || att->attgenerated)
continue;
4) This change is not required:
@@ -1015,7 +1110,6 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
walrcv_clear_result(res);
[1] - https://www.postgresql.org/message-id/CAD21AoB%3DDBVDNCGBja%2BsDa2-w9tsM7_E%3DZgyw2qYMR1R0FwDsg%40ma...
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-21 11:19 ` Amit Kapila <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-21 11:19 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Peter Smith <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 18, 2024 at 5:42 PM Shubham Khanna
<[email protected]> wrote:
> > >
> > > I have fixed all the given comments. The attached patches contain the
> > > required changes.
Review comments:
===============
1.
>
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
>
Is the second behavior introduced by the patch? If so, why?
2.
@@ -1213,7 +1207,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
...
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
It is better to combine the above conditions and write a comment on it.
3.
@@ -368,18 +379,50 @@ pub_collist_contains_invalid_column(Oid pubid,
Relation relation, List *ancestor
Anum_pg_publication_rel_prattrs,
&isnull);
- if (!isnull)
+ if (!isnull || !pubgencols)
{
int x;
Bitmapset *idattrs;
Bitmapset *columns = NULL;
- /* With REPLICA IDENTITY FULL, no column list is allowed. */
- if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
- result = true;
+ if (!isnull)
+ {
+ /* With REPLICA IDENTITY FULL, no column list is allowed. */
+ if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
+ result = true;
+
+ /* Transform the column list datum to a bitmapset. */
+ columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ }
+ else
+ {
+ TupleDesc desc = RelationGetDescr(relation);
+ int nliveatts = 0;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped)
+ continue;
+
+ nliveatts++;
+
+ if (att->attgenerated)
+ continue;
+
+ columns = bms_add_member(columns, i + 1);
+ }
- /* Transform the column list datum to a bitmapset. */
- columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ /* Return if all columns of the table will be replicated */
+ if (bms_num_members(columns) == nliveatts)
+ {
+ bms_free(columns);
+ ReleaseSysCache(tuple);
+ return false;
+ }
Won't this lead to traversing the entire column list for default cases
where publish_generated_columns would be false which could hurt the
update/delete's performance? Irrespective of that, it is better to
write some comments to explain this logic.
4. Some minimum parts of 0002 like the changes in
/doc/src/sgml/ref/create_publication.sgml should be part of 0001
patch. We can always add examples or more details in the docs as a
later patch.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-22 01:02 ` Peter Smith <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-22 01:02 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi SHubham, Here are my review comments for v40-0001 (code)
Please don't post a blanket response of "I have fixed all the
comments" response to this. Sometimes things get missed. Instead,
please reply done/not-done/whatever individually, so I can track the
changes properly.
======
src/backend/catalog/pg_publication.c
1.
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
This function no longer rejects generated columns from a publication
column list. So, now the function comment is wrong because it still
says "Checks for and raises an ERROR for any; unknown columns, system
columns, duplicate columns or generated columns."
NOTE: This was fixed already in v39-0001, but then broken again in
v40-0001 (???)
nit - also remove that semicolon (;) from the function comment.
======
src/backend/commands/publicationcmds.c
2.
- if (!isnull)
+ if (!isnull || !pubgencols)
{
int x;
Bitmapset *idattrs;
Bitmapset *columns = NULL;
- /* With REPLICA IDENTITY FULL, no column list is allowed. */
- if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
- result = true;
+ if (!isnull)
+ {
+ /* With REPLICA IDENTITY FULL, no column list is allowed. */
+ if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
+ result = true;
+
+ /* Transform the column list datum to a bitmapset. */
+ columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ }
+ else
+ {
+ TupleDesc desc = RelationGetDescr(relation);
+ int nliveatts = 0;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped)
+ continue;
+
+ nliveatts++;
+
+ if (att->attgenerated)
+ continue;
+
+ columns = bms_add_member(columns, i + 1);
+ }
- /* Transform the column list datum to a bitmapset. */
- columns = pub_collist_to_bitmapset(NULL, datum, NULL);
+ /* Return if all columns of the table will be replicated */
+ if (bms_num_members(columns) == nliveatts)
+ {
+ bms_free(columns);
+ ReleaseSysCache(tuple);
+ return false;
+ }
+ }
2a.
AFAIK this code was written to deal with Sawada-San's comment about
UPDATE/DELETE [1], but the logic is not very clear. It needs more
comments on the "else" part, the generated columns and how the new
logic solves Sawada-San's problem.
~
2b.
This function is now dealing both with 'publish_via_root' as well as
'publish_generated_columns'. I am suspicious that you may be handling
each of these parameters independently (i.e.. there is some early
return "Return if all columns of the table will be replicated") but
there might be some scenarios where a *combination* of pubviaroot and
gencols is not working as expected.
For example, there is a whole comment later about this: "If pubviaroot
is true, we are validating the column list of the parent table, but
the bitmap contains the replica identity information of the child
table." which I am not sure is being dealt with correctly. IMO there
need to be more test cases added for these tricky combination
scenarios to prove the code is good.
~
2c.
I expected to see some more REPLICA IDENTITY tests to reproduce the
problem scenario that Sawada-San reported. Where are those?
======
src/backend/replication/logical/tablesync.c
3. make_copy_attnamelist
Patch v38-0001 used to have a COPY error:
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a
generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
My most recent comment about this code was something along the lines
of "Make this detailed useful error message common if possible".
But, in v39 all this ERROR was removed (and it is still missing in
v40). Why? I did not see any other review comments asking for this to
be removed. AFAIK this was a correct error message for
not_generated->generated. Won't the current code now just silently
ignore/skip this, which is contrary behaviour to what normal
replication would do?
The recently deleted combo TAP tests could have detected this.
~~~
fetch_remote_table_info
4.
if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
There was a new 'server_version' variable added, so why not use it
here? In previous versions, we used to do so, but now since v39, the
code has reverted yet again. (???).
~~~
5.
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
Is this change OK? This fragment is guarded only by server_version >=
150000 (not 180000) so I was unsure. Can you explain why you made this
change, and why it is correct even for versions older than 180000?
~~~
6.
lrel->natts = natt;
-
walrcv_clear_result(res);
nit - this whitespace change has nothing to do with this patch. Remove it.
~~~
copy_table
7.
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
That mention about "when 'publish_generated_columns' is specified" is
not strictly true anymore, because the publication column list
overrides this parameter. It may be better to word this like below:
SUGGESTION
We also need to use this same COPY (SELECT ...) syntax when generated
columns are published, because copying generated columns is not
supported by the normal COPY.
======
src/backend/utils/cache/relcache.c
8.
/*
* Check if all columns are part of the REPLICA IDENTITY index or not.
*
- * If the publication is FOR ALL TABLES then it means the table has no
- * column list and we can skip the validation.
+ * If the publication is FOR ALL TABLES and publication includes
+ * generated columns then it means that all the table will replicate
+ * all columns and we can skip the validation.
*/
/table/tables/
Also, I think this can be re-worded to name the
'publish_generated_columns' parameter which might be more clear.
SUGGESTION:
If the publication is FOR ALL TABLES then column lists are not
possible. In this case, if 'publish_generated_columns' is true then
all table columns will be replicated, so the validation can be
skipped.
======
src/include/replication/logicalproto.h
9.
Bitmapset *attkeys; /* Bitmap of key columns */
+ bool *remotegenlist; /* Array to store whether each column is
+ * generated */
} LogicalRepRelation;
Since this is just a palloc'ed array same as the other fields like
'attnames' and 'atttyps', maybe it should be named and commented more
similarly to those? E.g more like:
bool *attremotegen; /* remote column is generated? */
======
src/test/regress/sql/publication.sql
10.
+-- Remove generate columns from column list, when
'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
typo /generate/generated/
======
src/test/subscription/t/100_bugs.pl
11.
- CREATE TABLE generated_cols (a int, b_gen int GENERATED ALWAYS AS (5
* a) STORED, c int);
+ CREATE TABLE generated_cols (a int, b_gen int, c int);
Hmm. But, should we be modifying tests in the "bugs" test file for a
new feature? If it is really necessary, then at least update the
comment to explain why.
~~~
12.
- CREATE TABLE generated_cols (a int, b_gen int GENERATED ALWAYS AS (5
* a) STORED, c int);
+ CREATE TABLE generated_cols (a int, b_gen int, c int);
Now you've ended up with a table called 'generated_cols' which has no
generated cols in it at all. Isn't this contrary to why this test case
existed in the first place? In other words, it feels a bit like a
hack. Perhaps you could have left all the tables as-is but just say
'publish_generated_columns=false'. But then that should be the default
parameter value anyhow, so TBH it is not clear to me why 100_bugs.pl
needed any changes at all. Can you give the reason?
======
PSA the nitpicks attachment which implements some of the cosmetic
suggestions from above.
======
[1] https://www.postgresql.org/message-id/CAD21AoB%3DDBVDNCGBja%2BsDa2-w9tsM7_E%3DZgyw2qYMR1R0FwDsg%40ma...
Kind Regards,
Peter Smith.
Fujitsu Australia.
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index e6e5506..eff876a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -500,8 +500,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
* valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * any unknown columns, system columns, or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 2f55fc5..b1ebefe 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -891,7 +891,7 @@ fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
@@ -1110,6 +1110,7 @@ fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
+
walrcv_clear_result(res);
/*
@@ -1288,9 +1289,8 @@ copy_table(Relation rel)
* SELECT query with OR'ed row filters for COPY.
*
* We also need to use this same COPY (SELECT ...) syntax when
- * 'publish_generated_columns' is specified as true and the remote
- * table has generated columns, because copy of generated columns is
- * not supported by the normal COPY.
+ * generated columns are published, because copy of generated columns
+ * is not supported by the normal COPY.
*/
int i = 0;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 80e11a3..0216b57 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5826,9 +5826,9 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
/*
* Check if all columns are part of the REPLICA IDENTITY index or not.
*
- * If the publication is FOR ALL TABLES and publication includes
- * generated columns then it means that all the table will replicate
- * all columns and we can skip the validation.
+ * If the publication is FOR ALL TABLES then column lists are not possible.
+ * In this case, if 'publish_generated_columns' is true then all table
+ * columns will be replicated, so the validation can be skipped.
*/
if (!(pubform->puballtables && pubform->pubgencols) &&
(pubform->pubupdate || pubform->pubdelete) &&
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index fc856d9..72943ef 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -1809,7 +1809,7 @@ ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
Tables:
"public.gencols" (a, gen1)
--- Remove generate columns from column list, when 'publish_generated_columns'=false
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
\dRp+ pub2
Publication pub2
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 454a03b..1ee322f 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -1140,7 +1140,7 @@ CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_colum
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
\dRp+ pub2
--- Remove generate columns from column list, when 'publish_generated_columns'=false
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
\dRp+ pub2
Attachments:
[text/plain] PS_NITPICKS_20241022_GENCOLS_V400001.txt (4.1K, ../../CAHut+Pu5GW-AdLGjbEFnS3+PS=Xq9KeCn7ch9BZQdzGom2gq2Q@mail.gmail.com/2-PS_NITPICKS_20241022_GENCOLS_V400001.txt)
download | inline diff:
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index e6e5506..eff876a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -500,8 +500,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
* valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * any unknown columns, system columns, or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 2f55fc5..b1ebefe 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -891,7 +891,7 @@ fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
@@ -1110,6 +1110,7 @@ fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
+
walrcv_clear_result(res);
/*
@@ -1288,9 +1289,8 @@ copy_table(Relation rel)
* SELECT query with OR'ed row filters for COPY.
*
* We also need to use this same COPY (SELECT ...) syntax when
- * 'publish_generated_columns' is specified as true and the remote
- * table has generated columns, because copy of generated columns is
- * not supported by the normal COPY.
+ * generated columns are published, because copy of generated columns
+ * is not supported by the normal COPY.
*/
int i = 0;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 80e11a3..0216b57 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5826,9 +5826,9 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
/*
* Check if all columns are part of the REPLICA IDENTITY index or not.
*
- * If the publication is FOR ALL TABLES and publication includes
- * generated columns then it means that all the table will replicate
- * all columns and we can skip the validation.
+ * If the publication is FOR ALL TABLES then column lists are not possible.
+ * In this case, if 'publish_generated_columns' is true then all table
+ * columns will be replicated, so the validation can be skipped.
*/
if (!(pubform->puballtables && pubform->pubgencols) &&
(pubform->pubupdate || pubform->pubdelete) &&
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index fc856d9..72943ef 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -1809,7 +1809,7 @@ ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
Tables:
"public.gencols" (a, gen1)
--- Remove generate columns from column list, when 'publish_generated_columns'=false
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
\dRp+ pub2
Publication pub2
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 454a03b..1ee322f 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -1140,7 +1140,7 @@ CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_colum
ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
\dRp+ pub2
--- Remove generate columns from column list, when 'publish_generated_columns'=false
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
ALTER PUBLICATION pub2 SET TABLE gencols(a);
\dRp+ pub2
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-22 07:28 ` Peter Smith <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-22 07:28 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi Shubham, here are some comments for v40-0003 (TAP) patch.
======
Combo tests
1.
+# =============================================================================
+# The following test cases exercise logical replication for the combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
Saying "where there is a generated column on one or both sides of the
pub/sub" was a valid comment back when all possible combinations were
tested. Now, if you are only going to test one case ("generated ->
normal") then that comment is plain wrong.
2.
Why have you removed "copy_data=true" from some comments, but not consistently?
======
DROP EXPRESSION test
3.
(from v39-0003)
+# A "normal -> generated" replication fails, reporting an error that the
+# subscriber side column is missing.
In v40-003 this comment was broken. In v39-0003 the above comment
mentioned about failing, but in v40-0003 that comment was removed. The
v39 comment was better, because that is the whole point of this test
-- that normal->generated would fail *unless* the DROP EXPRESSION
worked correctly and changed it to a normal->normal test.
Previously we had already "proved" normal->generated failed as
expected, but since you've removed all those combo tests, now all we
have is this comment to explain it.
~~~
4.
+# Insert data to verify replication.
+
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_alter VALUES (1,1), (2,2), (3,3)");
Remove unnecessary whitespace.
======
First "Testcase"
5.
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
That comment needs updating to reflect what this test case is *really*
doing. E.g., now you are testing tables without column lists as well
as tables with column lists.
~~~
6.
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int GENERATED ALWAYS AS (a
* 2) STORED);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int GENERATED ALWAYS AS (a
* 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen,
tab_gen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
Calling these tables 'gen_to_gen' makes no sense. Replication to a
subscriber-side generated column does not work, and anyway, your
subscriber-side tables do not have generated columns in them. Please
be very careful with the table names -- misleading names cause a lot
of unnecessary confusion.
~~~
7.
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen2 (a) VALUES (1), (1);
+));
+
It seems unusual to insert initial values 1,1, and then later insert
values 2,3. Wouldn't values 1,2, and then 3,4 be better?
~~~
8.
+# Create table and subscription with copy_data=true.
Sometimes you say "copy_data-true" in the comments and other times you
do not. I could not make sense of why the differences -- I guess
perhaps this was supposed to be a global replacement but some got
missed by mistake (???)
~~~
9.
Remove multiple unnecessary whitespace, in various places in this test.
~~~
10.
+# Initial sync test when publish_generated_columns=false.
It would be better if this test comments (and others like this one)
would also say more about what is the result expected and why.
~~~
11.
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|
+2|
+3|),
+ 'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
+);
There is not a column 'b' (I think now you called it "gen1") so the
comment is bogus. Please take care that comments are kept up-to-date
whenever you change the code.
~~~
12.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen2 incremental replication, when
publish_generated_columns=false'
+);
+
Here you should have a comment to explain that the expected result was
generated column 'gen1' should be replicated because it was specified
in a column list, so that overrides the
publish_generated_columns=false.
======
Second "Testcase"
13.
All the above comments also apply to this second test case:
e.g. the "Testcase" comment needed updating.
e.g. table names like 'gen_to_gen' make no sense.
e.g. the initial data values 1,1 seem strange to me.
e.g. some comments have spurious "copy_data = true" and some do not.
e.g. unnecessary blank lines.
e.g. not enough comments describing what are the expected results and why.
e.g. multiple bogus mentions of a column 'b', when there is no such column name
======
FYI - The attached nitpicks just show some of the blank lines I
removed; nothing else.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index ff44c87..24134fa 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -239,7 +239,6 @@ $node_subscriber->safe_psql('postgres',
"ALTER TABLE tab_alter ALTER COLUMN b DROP EXPRESSION");
# Insert data to verify replication.
-
$node_publisher->safe_psql('postgres',
"INSERT INTO tab_alter VALUES (1,1), (2,2), (3,3)");
@@ -297,7 +296,6 @@ $node_subscriber->safe_psql(
# Wait for initial sync.
$node_subscriber->wait_for_subscription_sync;
-
$node_publisher->wait_for_catchup('sub1');
# Initial sync test when publish_generated_columns=false.
@@ -306,7 +304,6 @@ $result = $node_subscriber->safe_psql('postgres',
is( $result, qq(1|
1|),
'tab_gen_to_gen initial sync, when publish_generated_columns=false');
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen2 ORDER BY a");
is( $result, qq(|2
@@ -331,7 +328,6 @@ is( $result, qq(1|
3|),
'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
);
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen2 ORDER BY a");
is( $result, qq(|2
@@ -376,7 +372,6 @@ $node_subscriber->safe_psql(
# Wait for initial sync.
$node_subscriber->wait_for_subscription_sync;
-
$node_publisher->wait_for_catchup('sub1');
# Initial sync test when publish_generated_columns=true.
@@ -385,7 +380,6 @@ $result = $node_subscriber->safe_psql('postgres',
is( $result, qq(1|2
1|2),
'tab_gen_to_gen3 initial sync, when publish_generated_columns=true');
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen4 ORDER BY a");
is( $result, qq(|2
@@ -411,7 +405,6 @@ is( $result, qq(1|2
3|6),
'tab_gen_to_gen3 incremental replication, when publish_generated_columns=true'
);
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen4 ORDER BY a");
is( $result, qq(|2
Attachments:
[text/plain] PS_NITPICKS_20241022_GENCOLS_V400003.txt (2.0K, ../../CAHut+Ps47=CwrWW2NX135DXJBZd5Bm4m-6xFwAzep2yeT567cA@mail.gmail.com/2-PS_NITPICKS_20241022_GENCOLS_V400003.txt)
download | inline diff:
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index ff44c87..24134fa 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -239,7 +239,6 @@ $node_subscriber->safe_psql('postgres',
"ALTER TABLE tab_alter ALTER COLUMN b DROP EXPRESSION");
# Insert data to verify replication.
-
$node_publisher->safe_psql('postgres',
"INSERT INTO tab_alter VALUES (1,1), (2,2), (3,3)");
@@ -297,7 +296,6 @@ $node_subscriber->safe_psql(
# Wait for initial sync.
$node_subscriber->wait_for_subscription_sync;
-
$node_publisher->wait_for_catchup('sub1');
# Initial sync test when publish_generated_columns=false.
@@ -306,7 +304,6 @@ $result = $node_subscriber->safe_psql('postgres',
is( $result, qq(1|
1|),
'tab_gen_to_gen initial sync, when publish_generated_columns=false');
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen2 ORDER BY a");
is( $result, qq(|2
@@ -331,7 +328,6 @@ is( $result, qq(1|
3|),
'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
);
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen2 ORDER BY a");
is( $result, qq(|2
@@ -376,7 +372,6 @@ $node_subscriber->safe_psql(
# Wait for initial sync.
$node_subscriber->wait_for_subscription_sync;
-
$node_publisher->wait_for_catchup('sub1');
# Initial sync test when publish_generated_columns=true.
@@ -385,7 +380,6 @@ $result = $node_subscriber->safe_psql('postgres',
is( $result, qq(1|2
1|2),
'tab_gen_to_gen3 initial sync, when publish_generated_columns=true');
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen4 ORDER BY a");
is( $result, qq(|2
@@ -411,7 +405,6 @@ is( $result, qq(1|2
3|6),
'tab_gen_to_gen3 incremental replication, when publish_generated_columns=true'
);
-
$result = $node_subscriber->safe_psql('postgres',
"SELECT * FROM tab_gen_to_gen4 ORDER BY a");
is( $result, qq(|2
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-23 05:00 ` Peter Smith <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-23 05:00 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Recently (~ version v39/v40) some changes to 'get_publications_str'
calls got removed from this patchset because it was decided it was
really a separate problem, unrelated to this generated columns
feature.
FYI - I've started a new thread "Refactor to use common function
'get_publications_str'" [1] to address it separately.
======
[1] https://www.postgresql.org/message-id/[email protected]...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-17 10:29 ` vignesh C <[email protected]>
2024-10-18 12:15 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-17 10:29 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, 16 Oct 2024 at 23:25, Shubham Khanna
<[email protected]> wrote:
>
> On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
> >
> > On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> > >
> > > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > > >
> > > > Hi Shubham, here are my review comments for v36-0001.
> > > >
> > > > ======
> > > > 1. General - merge patches
> > > >
> > > > It is long past due when patches 0001 and 0002 should've been merged.
> > > > AFAIK the split was only because historically these parts had
> > > > different authors. But, keeping them separated is not helpful anymore.
> > > >
> > > > ======
> > > > src/backend/catalog/pg_publication.c
> > > >
> > > > 2.
> > > > Bitmapset *
> > > > -pub_collist_validate(Relation targetrel, List *columns)
> > > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > > >
> > > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > > so it should also be removed.
> > > >
> > > > ======
> > > > src/backend/replication/pgoutput/pgoutput.c
> > > >
> > > > 3.
> > > > /*
> > > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > > - * if there are no column lists (even if other publications have a
> > > > - * list).
> > > > + * To handle cases where the publish_generated_columns option isn't
> > > > + * specified for all tables in a publication, we must create a column
> > > > + * list that excludes generated columns. So, the publisher will not
> > > > + * replicate the generated columns.
> > > > */
> > > > - if (!pub->alltables)
> > > > + if (!(pub->alltables && pub->pubgencols))
> > > >
> > > > I still found that comment hard to understand. Does this mean to say
> > > > something like:
> > > >
> > > > ------
> > > > Process potential column lists for the following cases:
> > > >
> > > > a. Any publication that is not FOR ALL TABLES.
> > > >
> > > > b. When the publication is FOR ALL TABLES and
> > > > 'publish_generated_columns' is false.
> > > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > > so all columns will be replicated by default. However, if
> > > > 'publish_generated_columns' is set to false, column lists must still
> > > > be created to exclude any generated columns from being published
> > > > ------
> > > >
> > > > ======
> > > > src/test/regress/sql/publication.sql
> > > >
> > > > 4.
> > > > +SET client_min_messages = 'WARNING';
> > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > >
> > > > AFAIK you don't need to keep changing 'client_min_messages',
> > > > particularly now that you've removed the WARNING message that was
> > > > previously emitted.
> > > >
> > > > ~
> > > >
> > > > 5.
> > > > nit - minor comment changes.
> > > >
> > > > ======
> > > > Please refer to the attachment which implements any nits from above.
> > > >
> > >
> > > I have fixed all the given comments. Also, I have created a new 0003
> > > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > > planning to merge 0001 and 0003 patches once they will get fixed.
> > > The attached patches contain the required changes.
> >
> > Few comments:
> > 1) Since we are no longer throwing an error for generated columns, the
> > function header comments also need to be updated accordingly " Checks
> > for and raises an ERROR for any; unknown columns, system columns,
> > duplicate columns or generated columns."
> > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > - ereport(ERROR,
> > -
> > errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > - errmsg("cannot use generated
> > column \"%s\" in publication column list",
> > - colname));
> > -
> >
> > 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> > ALTER PUBLICATION ... SET (
> > postgres=# alter publication pub2 set (PUBLISH
> > PUBLISH PUBLISH_VIA_PARTITION_ROOT
> >
> > 3) I was able to compile without this include, may be this is not required:
> > --- a/src/backend/replication/logical/tablesync.c
> > +++ b/src/backend/replication/logical/tablesync.c
> > @@ -118,6 +118,7 @@
> > #include "utils/builtins.h"
> > #include "utils/lsyscache.h"
> > #include "utils/memutils.h"
> > +#include "utils/rel.h"
> >
> > 4) You can include "\dRp+ pubname" after each of the create/alter
> > publication to verify the columns that will be published:
> > +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> > +-- different scenarios with/without generated columns in column lists.
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > +
> > +-- Generated columns in column list, when 'publish_generated_columns'=false
> > +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> > (publish_generated_columns=false);
> >
> > +-- Generated columns in column list, when 'publish_generated_columns'=true
> > +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> > (publish_generated_columns=true);
> > +
> > +-- Generated columns in column list, then set
> > 'publication_generate_columns'=false
> > +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> > +
> > +-- Remove generate columns from column list, when
> > 'publish_generated_columns'=false
> > +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> > +
> > +-- Add generated columns in column list, when 'publish_generated_columns'=false
> > +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
> >
>
> I have fixed all the given comments. The attached patches contain the
> required changes.
Few comments:
1) File mode change is not required:
src/test/subscription/t/011_generated.pl | 354 +++++++++++++++++++++++
1 file changed, 354 insertions(+)
mode change 100644 => 100755 src/test/subscription/t/011_generated.pl
diff --git a/src/test/subscription/t/011_generated.pl
b/src/test/subscription/t/011_generated.pl
old mode 100644
new mode 100755
index 8b2e5f4708..d1f2718078
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
2) Here copy_data=true looks obvious no need to mention again and
again in comments:
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION
'$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH
(copy_data = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION
'$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH
(copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false and copy_data=true.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true and copy_data=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
3) The database test_pgc_true and also can be cleaned as it is not
required after this:
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
4) There is no error message verification in this test, let's add the
error verification:
+# =============================================================================
+# Misc test.
+#
+# A "normal -> generated" replication fails, reporting an error that the
+# subscriber side column is missing.
+#
+# In this test case we use DROP EXPRESSION to change the subscriber generated
+# column into a normal column, then verify replication works ok.
+# =============================================================================
5)
5.a) If possible have one regular column and one generated column in the tables
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int GENERATED
ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2)
STORED);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int GENERATED
ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2)
STORED);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int GENERATED
ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2)
STORED);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen(a, b, gen2),
gen_to_nogen2, nogen_to_gen2(gen1) WITH
(publish_generated_columns=false);
+));
5.b) Try to have same columns in all the tables
6) These are inserting two records:
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen VALUES (2), (3);
+ INSERT INTO gen_to_nogen2 VALUES (2), (3);
+ INSERT INTO nogen_to_gen2 VALUES (2), (3);
+));
I felt you wanted this to be:
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO gen_to_nogen VALUES (2, 3);
+ INSERT INTO gen_to_nogen2 VALUES (2, 3);
+ INSERT INTO nogen_to_gen2 VALUES (2, 3);
+));
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 10:29 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-18 12:15 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-18 12:15 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, Oct 17, 2024 at 3:59 PM vignesh C <[email protected]> wrote:
>
> On Wed, 16 Oct 2024 at 23:25, Shubham Khanna
> <[email protected]> wrote:
> >
> > On Wed, Oct 9, 2024 at 9:08 AM vignesh C <[email protected]> wrote:
> > >
> > > On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> > > >
> > > > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > > > >
> > > > > Hi Shubham, here are my review comments for v36-0001.
> > > > >
> > > > > ======
> > > > > 1. General - merge patches
> > > > >
> > > > > It is long past due when patches 0001 and 0002 should've been merged.
> > > > > AFAIK the split was only because historically these parts had
> > > > > different authors. But, keeping them separated is not helpful anymore.
> > > > >
> > > > > ======
> > > > > src/backend/catalog/pg_publication.c
> > > > >
> > > > > 2.
> > > > > Bitmapset *
> > > > > -pub_collist_validate(Relation targetrel, List *columns)
> > > > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > > > >
> > > > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > > > so it should also be removed.
> > > > >
> > > > > ======
> > > > > src/backend/replication/pgoutput/pgoutput.c
> > > > >
> > > > > 3.
> > > > > /*
> > > > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > > > - * if there are no column lists (even if other publications have a
> > > > > - * list).
> > > > > + * To handle cases where the publish_generated_columns option isn't
> > > > > + * specified for all tables in a publication, we must create a column
> > > > > + * list that excludes generated columns. So, the publisher will not
> > > > > + * replicate the generated columns.
> > > > > */
> > > > > - if (!pub->alltables)
> > > > > + if (!(pub->alltables && pub->pubgencols))
> > > > >
> > > > > I still found that comment hard to understand. Does this mean to say
> > > > > something like:
> > > > >
> > > > > ------
> > > > > Process potential column lists for the following cases:
> > > > >
> > > > > a. Any publication that is not FOR ALL TABLES.
> > > > >
> > > > > b. When the publication is FOR ALL TABLES and
> > > > > 'publish_generated_columns' is false.
> > > > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > > > so all columns will be replicated by default. However, if
> > > > > 'publish_generated_columns' is set to false, column lists must still
> > > > > be created to exclude any generated columns from being published
> > > > > ------
> > > > >
> > > > > ======
> > > > > src/test/regress/sql/publication.sql
> > > > >
> > > > > 4.
> > > > > +SET client_min_messages = 'WARNING';
> > > > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > > >
> > > > > AFAIK you don't need to keep changing 'client_min_messages',
> > > > > particularly now that you've removed the WARNING message that was
> > > > > previously emitted.
> > > > >
> > > > > ~
> > > > >
> > > > > 5.
> > > > > nit - minor comment changes.
> > > > >
> > > > > ======
> > > > > Please refer to the attachment which implements any nits from above.
> > > > >
> > > >
> > > > I have fixed all the given comments. Also, I have created a new 0003
> > > > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > > > planning to merge 0001 and 0003 patches once they will get fixed.
> > > > The attached patches contain the required changes.
> > >
> > > Few comments:
> > > 1) Since we are no longer throwing an error for generated columns, the
> > > function header comments also need to be updated accordingly " Checks
> > > for and raises an ERROR for any; unknown columns, system columns,
> > > duplicate columns or generated columns."
> > > - if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> > > - ereport(ERROR,
> > > -
> > > errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> > > - errmsg("cannot use generated
> > > column \"%s\" in publication column list",
> > > - colname));
> > > -
> > >
> > > 2) Tab completion missing for "PUBLISH_GENERATED_COLUMNS" option in
> > > ALTER PUBLICATION ... SET (
> > > postgres=# alter publication pub2 set (PUBLISH
> > > PUBLISH PUBLISH_VIA_PARTITION_ROOT
> > >
> > > 3) I was able to compile without this include, may be this is not required:
> > > --- a/src/backend/replication/logical/tablesync.c
> > > +++ b/src/backend/replication/logical/tablesync.c
> > > @@ -118,6 +118,7 @@
> > > #include "utils/builtins.h"
> > > #include "utils/lsyscache.h"
> > > #include "utils/memutils.h"
> > > +#include "utils/rel.h"
> > >
> > > 4) You can include "\dRp+ pubname" after each of the create/alter
> > > publication to verify the columns that will be published:
> > > +-- Test the 'publish_generated_columns' parameter enabled or disabled for
> > > +-- different scenarios with/without generated columns in column lists.
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > > +
> > > +-- Generated columns in column list, when 'publish_generated_columns'=false
> > > +CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH
> > > (publish_generated_columns=false);
> > >
> > > +-- Generated columns in column list, when 'publish_generated_columns'=true
> > > +CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH
> > > (publish_generated_columns=true);
> > > +
> > > +-- Generated columns in column list, then set
> > > 'publication_generate_columns'=false
> > > +ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
> > > +
> > > +-- Remove generate columns from column list, when
> > > 'publish_generated_columns'=false
> > > +ALTER PUBLICATION pub2 SET TABLE gencols(a);
> > > +
> > > +-- Add generated columns in column list, when 'publish_generated_columns'=false
> > > +ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
> > >
> >
> > I have fixed all the given comments. The attached patches contain the
> > required changes.
>
> Few comments:
> 1) File mode change is not required:
> src/test/subscription/t/011_generated.pl | 354 +++++++++++++++++++++++
> 1 file changed, 354 insertions(+)
> mode change 100644 => 100755 src/test/subscription/t/011_generated.pl
>
> diff --git a/src/test/subscription/t/011_generated.pl
> b/src/test/subscription/t/011_generated.pl
> old mode 100644
> new mode 100755
> index 8b2e5f4708..d1f2718078
> --- a/src/test/subscription/t/011_generated.pl
> +++ b/src/test/subscription/t/011_generated.pl
>
> 2) Here copy_data=true looks obvious no need to mention again and
> again in comments:
> +# Create table and subscription with copy_data=true.
> +$node_subscriber->safe_psql(
> + 'postgres', qq(
> + CREATE TABLE tab_gen_to_nogen (a int, b int);
> + CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION
> '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH
> (copy_data = true);
> +));
> +
> +# Create table and subscription with copy_data=true.
> +$node_subscriber->safe_psql(
> + 'test_pgc_true', qq(
> + CREATE TABLE tab_gen_to_nogen (a int, b int);
> + CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION
> '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH
> (copy_data = true);
> +));
> +
> +# Wait for initial sync.
> +$node_subscriber->wait_for_subscription_sync($node_publisher,
> + 'regress_sub1_gen_to_nogen', 'postgres');
> +$node_subscriber->wait_for_subscription_sync($node_publisher,
> + 'regress_sub2_gen_to_nogen', 'test_pgc_true');
> +
> +# Initial sync test when publish_generated_columns=false and copy_data=true.
> +# Verify that column 'b' is not replicated.
> +$result = $node_subscriber->safe_psql('postgres',
> + "SELECT a, b FROM tab_gen_to_nogen");
> +is( $result, qq(1|
> +2|
> +3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
> +
> +# Initial sync test when publish_generated_columns=true and copy_data=true.
> +$result = $node_subscriber->safe_psql('test_pgc_true',
> + "SELECT a, b FROM tab_gen_to_nogen");
> +is( $result, qq(1|2
> +2|4
> +3|6),
> + 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
>
> 3) The database test_pgc_true and also can be cleaned as it is not
> required after this:
> +# cleanup
> +$node_subscriber->safe_psql('postgres',
> + "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
> +$node_subscriber->safe_psql('test_pgc_true',
> + "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
> +$node_publisher->safe_psql(
> + 'postgres', qq(
> + DROP PUBLICATION regress_pub1_gen_to_nogen;
> + DROP PUBLICATION regress_pub2_gen_to_nogen;
> +));
>
> 4) There is no error message verification in this test, let's add the
> error verification:
> +# =============================================================================
> +# Misc test.
> +#
> +# A "normal -> generated" replication fails, reporting an error that the
> +# subscriber side column is missing.
> +#
> +# In this test case we use DROP EXPRESSION to change the subscriber generated
> +# column into a normal column, then verify replication works ok.
> +# =============================================================================
>
> 5)
> 5.a) If possible have one regular column and one generated column in the tables
> +# --------------------------------------------------
> +# Testcase: Publisher replicates the column list data including generated
> +# columns even though publish_generated_columns option is false.
> +# --------------------------------------------------
> +
> +# Create table and publications.
> +$node_publisher->safe_psql(
> + 'postgres', qq(
> + CREATE TABLE gen_to_nogen (a int, b int, gen1 int GENERATED
> ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2)
> STORED);
> + CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int GENERATED
> ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2)
> STORED);
> + CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int GENERATED
> ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2)
> STORED);
> + CREATE PUBLICATION pub1 FOR table gen_to_nogen(a, b, gen2),
> gen_to_nogen2, nogen_to_gen2(gen1) WITH
> (publish_generated_columns=false);
> +));
>
> 5.b) Try to have same columns in all the tables
>
> 6) These are inserting two records:
> +# Insert data to verify incremental replication
> +$node_publisher->safe_psql(
> + 'postgres', qq(
> + INSERT INTO gen_to_nogen VALUES (2), (3);
> + INSERT INTO gen_to_nogen2 VALUES (2), (3);
> + INSERT INTO nogen_to_gen2 VALUES (2), (3);
> +));
>
> I felt you wanted this to be:
> +# Insert data to verify incremental replication
> +$node_publisher->safe_psql(
> + 'postgres', qq(
> + INSERT INTO gen_to_nogen VALUES (2, 3);
> + INSERT INTO gen_to_nogen2 VALUES (2, 3);
> + INSERT INTO nogen_to_gen2 VALUES (2, 3);
> +));
I have fixed all the comments and posted the v40 patches for them.
Please refer to the updated v40 Patches here in [1]. See [1] for the
changes added.
[1] https://www.postgresql.org/message-id/CAHv8RjLviXAWtB3Kcn1A1jPpqORpkNay1y2U%2B55K64sqwCdrGw%40mail.g...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-09 05:30 ` vignesh C <[email protected]>
2024-10-16 17:59 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
5 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-09 05:30 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
>
> On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> >
> > Hi Shubham, here are my review comments for v36-0001.
> >
> > ======
> > 1. General - merge patches
> >
> > It is long past due when patches 0001 and 0002 should've been merged.
> > AFAIK the split was only because historically these parts had
> > different authors. But, keeping them separated is not helpful anymore.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 2.
> > Bitmapset *
> > -pub_collist_validate(Relation targetrel, List *columns)
> > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> >
> > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > so it should also be removed.
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 3.
> > /*
> > - * If the publication is FOR ALL TABLES then it is treated the same as
> > - * if there are no column lists (even if other publications have a
> > - * list).
> > + * To handle cases where the publish_generated_columns option isn't
> > + * specified for all tables in a publication, we must create a column
> > + * list that excludes generated columns. So, the publisher will not
> > + * replicate the generated columns.
> > */
> > - if (!pub->alltables)
> > + if (!(pub->alltables && pub->pubgencols))
> >
> > I still found that comment hard to understand. Does this mean to say
> > something like:
> >
> > ------
> > Process potential column lists for the following cases:
> >
> > a. Any publication that is not FOR ALL TABLES.
> >
> > b. When the publication is FOR ALL TABLES and
> > 'publish_generated_columns' is false.
> > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > so all columns will be replicated by default. However, if
> > 'publish_generated_columns' is set to false, column lists must still
> > be created to exclude any generated columns from being published
> > ------
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 4.
> > +SET client_min_messages = 'WARNING';
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> >
> > AFAIK you don't need to keep changing 'client_min_messages',
> > particularly now that you've removed the WARNING message that was
> > previously emitted.
> >
> > ~
> >
> > 5.
> > nit - minor comment changes.
> >
> > ======
> > Please refer to the attachment which implements any nits from above.
> >
>
> I have fixed all the given comments. Also, I have created a new 0003
> patch for the TAP-Tests related to the '011_generated.pl' file. I am
> planning to merge 0001 and 0003 patches once they will get fixed.
> The attached patches contain the required changes.
There is inconsistency in replication when a generated column is
specified in the column list. The generated column data is not
replicated during initial sync whereas it is getting replicated during
incremental sync:
-- publisher
CREATE TABLE t1(c1 int, c2 int GENERATED ALWAYS AS (c1 * 2) STORED)
INSERT INTO t1 VALUES (1);
CREATE PUBLICATION pub1 for table t1(c1, c2);
--subscriber
CREATE TABLE t1(c1 int, c2 int)
CREATE SUBSCRIPTION sub1 connection 'dbname=postgres host=localhost
port=5432' PUBLICATION pub1;
-- Generate column data is not synced during initial sync
postgres=# select * from t1;
c1 | c2
----+----
1 |
(1 row)
-- publisher
INSERT INTO t1 VALUES (2);
-- Whereas generated column data is synced during incremental sync
postgres=# select * from t1;
c1 | c2
----+----
1 |
2 | 4
(2 rows)
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 05:30 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-16 17:59 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-16 17:59 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 9, 2024 at 11:00 AM vignesh C <[email protected]> wrote:
>
> On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> >
> > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > >
> > > Hi Shubham, here are my review comments for v36-0001.
> > >
> > > ======
> > > 1. General - merge patches
> > >
> > > It is long past due when patches 0001 and 0002 should've been merged.
> > > AFAIK the split was only because historically these parts had
> > > different authors. But, keeping them separated is not helpful anymore.
> > >
> > > ======
> > > src/backend/catalog/pg_publication.c
> > >
> > > 2.
> > > Bitmapset *
> > > -pub_collist_validate(Relation targetrel, List *columns)
> > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > >
> > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > so it should also be removed.
> > >
> > > ======
> > > src/backend/replication/pgoutput/pgoutput.c
> > >
> > > 3.
> > > /*
> > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > - * if there are no column lists (even if other publications have a
> > > - * list).
> > > + * To handle cases where the publish_generated_columns option isn't
> > > + * specified for all tables in a publication, we must create a column
> > > + * list that excludes generated columns. So, the publisher will not
> > > + * replicate the generated columns.
> > > */
> > > - if (!pub->alltables)
> > > + if (!(pub->alltables && pub->pubgencols))
> > >
> > > I still found that comment hard to understand. Does this mean to say
> > > something like:
> > >
> > > ------
> > > Process potential column lists for the following cases:
> > >
> > > a. Any publication that is not FOR ALL TABLES.
> > >
> > > b. When the publication is FOR ALL TABLES and
> > > 'publish_generated_columns' is false.
> > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > so all columns will be replicated by default. However, if
> > > 'publish_generated_columns' is set to false, column lists must still
> > > be created to exclude any generated columns from being published
> > > ------
> > >
> > > ======
> > > src/test/regress/sql/publication.sql
> > >
> > > 4.
> > > +SET client_min_messages = 'WARNING';
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > >
> > > AFAIK you don't need to keep changing 'client_min_messages',
> > > particularly now that you've removed the WARNING message that was
> > > previously emitted.
> > >
> > > ~
> > >
> > > 5.
> > > nit - minor comment changes.
> > >
> > > ======
> > > Please refer to the attachment which implements any nits from above.
> > >
> >
> > I have fixed all the given comments. Also, I have created a new 0003
> > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > planning to merge 0001 and 0003 patches once they will get fixed.
> > The attached patches contain the required changes.
>
> There is inconsistency in replication when a generated column is
> specified in the column list. The generated column data is not
> replicated during initial sync whereas it is getting replicated during
> incremental sync:
> -- publisher
> CREATE TABLE t1(c1 int, c2 int GENERATED ALWAYS AS (c1 * 2) STORED)
> INSERT INTO t1 VALUES (1);
> CREATE PUBLICATION pub1 for table t1(c1, c2);
>
> --subscriber
> CREATE TABLE t1(c1 int, c2 int)
> CREATE SUBSCRIPTION sub1 connection 'dbname=postgres host=localhost
> port=5432' PUBLICATION pub1;
>
> -- Generate column data is not synced during initial sync
> postgres=# select * from t1;
> c1 | c2
> ----+----
> 1 |
> (1 row)
>
> -- publisher
> INSERT INTO t1 VALUES (2);
>
> -- Whereas generated column data is synced during incremental sync
> postgres=# select * from t1;
> c1 | c2
> ----+----
> 1 |
> 2 | 4
> (2 rows)
>
There was an issue for this scenario:
CREATE TABLE t1(c1 int, c2 int GENERATED ALWAYS AS (c1 * 2) STORED)
create publication pub1 for table t1(c1, c2)
In this case included_cols was getting set to NULL.
Changed it to get included_cols as it is instead of replacing with
NULL and changed the condition to:
if (server_version >= 180000)
{
remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
/*
* If the column is generated and neither the generated column
* option is specified nor it appears in the column list, we will
* skip it.
*/
if (remotegenlist[natt] && !has_pub_with_pubgencols && !included_cols)
{
ExecClearTuple(slot);
continue;
}
}
I will further think if there is a better solution for this.
Please refer to the updated v39 Patches here in [1]. See [1] for the
changes added.
[1] https://www.postgresql.org/message-id/CAHv8RjLjb%2B98i5ZQUphivxdOZ3hSGLfq2SiWQetUvk8zGyAQwQ%40mail.g...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-09 06:22 ` vignesh C <[email protected]>
2024-10-16 18:02 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
5 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-09 06:22 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
>
> On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> >
> > Hi Shubham, here are my review comments for v36-0001.
> >
> > ======
> > 1. General - merge patches
> >
> > It is long past due when patches 0001 and 0002 should've been merged.
> > AFAIK the split was only because historically these parts had
> > different authors. But, keeping them separated is not helpful anymore.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 2.
> > Bitmapset *
> > -pub_collist_validate(Relation targetrel, List *columns)
> > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> >
> > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > so it should also be removed.
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 3.
> > /*
> > - * If the publication is FOR ALL TABLES then it is treated the same as
> > - * if there are no column lists (even if other publications have a
> > - * list).
> > + * To handle cases where the publish_generated_columns option isn't
> > + * specified for all tables in a publication, we must create a column
> > + * list that excludes generated columns. So, the publisher will not
> > + * replicate the generated columns.
> > */
> > - if (!pub->alltables)
> > + if (!(pub->alltables && pub->pubgencols))
> >
> > I still found that comment hard to understand. Does this mean to say
> > something like:
> >
> > ------
> > Process potential column lists for the following cases:
> >
> > a. Any publication that is not FOR ALL TABLES.
> >
> > b. When the publication is FOR ALL TABLES and
> > 'publish_generated_columns' is false.
> > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > so all columns will be replicated by default. However, if
> > 'publish_generated_columns' is set to false, column lists must still
> > be created to exclude any generated columns from being published
> > ------
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 4.
> > +SET client_min_messages = 'WARNING';
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> >
> > AFAIK you don't need to keep changing 'client_min_messages',
> > particularly now that you've removed the WARNING message that was
> > previously emitted.
> >
> > ~
> >
> > 5.
> > nit - minor comment changes.
> >
> > ======
> > Please refer to the attachment which implements any nits from above.
> >
>
> I have fixed all the given comments. Also, I have created a new 0003
> patch for the TAP-Tests related to the '011_generated.pl' file. I am
> planning to merge 0001 and 0003 patches once they will get fixed.
> The attached patches contain the required changes.
Few comments:
1) I felt this change need not be part of this patch, if required it
can be proposed as a separate patch:
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names,
quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
2) These two statements can be combined in to single appendStringInfo:
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid =
pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped",
lrel->remoteid);
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
-
(walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
3) In which scenario this will be hit:
+ /*
+ * Construct column list for COPY, excluding columns that are
subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+
makeString(rel->remoterel.attnames[i]));
+ }
As in case of publisher having non generated columns:
CREATE TABLE t1(c1 int, c2 int)
and subscriber having generated columns:
CREATE TABLE t1(c1 int, c2 int GENERATED ALWAYS AS (c1 * 2) STORED)
We throw an error much earlier at
logicalrep_rel_open->logicalrep_report_missing_attrs saying:
ERROR: logical replication target relation "public.t1" is missing
replicated column: "c2"
4) To simplify the code and reduce complexity, we can refactor the
error checks to be included within the fetch_remote_table_info
function. This way, the remotegenlist will not need to be prepared and
passed to make_copy_attnamelist:
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
-
makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+
NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated
column has same name
+ * as a non-generated column in the
corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical
replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but
corresponding column on source relation is not a generated column",
+
rel->remoterel.nspname, rel->remoterel.relname,
NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a
generated column in the
+ * subscription table. Later, we use this
information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 06:22 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-16 18:02 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-16 18:02 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 9, 2024 at 11:52 AM vignesh C <[email protected]> wrote:
>
> On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> >
> > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > >
> > > Hi Shubham, here are my review comments for v36-0001.
> > >
> > > ======
> > > 1. General - merge patches
> > >
> > > It is long past due when patches 0001 and 0002 should've been merged.
> > > AFAIK the split was only because historically these parts had
> > > different authors. But, keeping them separated is not helpful anymore.
> > >
> > > ======
> > > src/backend/catalog/pg_publication.c
> > >
> > > 2.
> > > Bitmapset *
> > > -pub_collist_validate(Relation targetrel, List *columns)
> > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > >
> > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > so it should also be removed.
> > >
> > > ======
> > > src/backend/replication/pgoutput/pgoutput.c
> > >
> > > 3.
> > > /*
> > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > - * if there are no column lists (even if other publications have a
> > > - * list).
> > > + * To handle cases where the publish_generated_columns option isn't
> > > + * specified for all tables in a publication, we must create a column
> > > + * list that excludes generated columns. So, the publisher will not
> > > + * replicate the generated columns.
> > > */
> > > - if (!pub->alltables)
> > > + if (!(pub->alltables && pub->pubgencols))
> > >
> > > I still found that comment hard to understand. Does this mean to say
> > > something like:
> > >
> > > ------
> > > Process potential column lists for the following cases:
> > >
> > > a. Any publication that is not FOR ALL TABLES.
> > >
> > > b. When the publication is FOR ALL TABLES and
> > > 'publish_generated_columns' is false.
> > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > so all columns will be replicated by default. However, if
> > > 'publish_generated_columns' is set to false, column lists must still
> > > be created to exclude any generated columns from being published
> > > ------
> > >
> > > ======
> > > src/test/regress/sql/publication.sql
> > >
> > > 4.
> > > +SET client_min_messages = 'WARNING';
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > >
> > > AFAIK you don't need to keep changing 'client_min_messages',
> > > particularly now that you've removed the WARNING message that was
> > > previously emitted.
> > >
> > > ~
> > >
> > > 5.
> > > nit - minor comment changes.
> > >
> > > ======
> > > Please refer to the attachment which implements any nits from above.
> > >
> >
> > I have fixed all the given comments. Also, I have created a new 0003
> > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > planning to merge 0001 and 0003 patches once they will get fixed.
> > The attached patches contain the required changes.
>
> Few comments:
> 1) I felt this change need not be part of this patch, if required it
> can be proposed as a separate patch:
> + if (server_version >= 150000)
> {
> WalRcvExecResult *pubres;
> TupleTableSlot *tslot;
> Oid attrsRow[] = {INT2VECTOROID};
> - StringInfoData pub_names;
> -
> - initStringInfo(&pub_names);
> - foreach(lc, MySubscription->publications)
> - {
> - if (foreach_current_index(lc) > 0)
> - appendStringInfoString(&pub_names, ", ");
> - appendStringInfoString(&pub_names,
> quote_literal_cstr(strVal(lfirst(lc))));
> - }
> + StringInfo pub_names = makeStringInfo();
>
> 2) These two statements can be combined in to single appendStringInfo:
> + appendStringInfo(&cmd,
> " FROM pg_catalog.pg_attribute a"
> " LEFT JOIN pg_catalog.pg_index i"
> " ON (i.indexrelid =
> pg_get_replica_identity_index(%u))"
> " WHERE a.attnum > 0::pg_catalog.int2"
> - " AND NOT a.attisdropped %s"
> + " AND NOT a.attisdropped",
> lrel->remoteid);
> +
> + appendStringInfo(&cmd,
> " AND a.attrelid = %u"
> " ORDER BY a.attnum",
> - lrel->remoteid,
> -
> (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
> - "AND a.attgenerated = ''" : ""),
> lrel->remoteid);
>
> 3) In which scenario this will be hit:
> + /*
> + * Construct column list for COPY, excluding columns that are
> subscription
> + * table generated columns.
> + */
> + for (int i = 0; i < rel->remoterel.natts; i++)
> + {
> + if (!localgenlist[i])
> + attnamelist = lappend(attnamelist,
> +
> makeString(rel->remoterel.attnames[i]));
> + }
>
> As in case of publisher having non generated columns:
> CREATE TABLE t1(c1 int, c2 int)
> and subscriber having generated columns:
> CREATE TABLE t1(c1 int, c2 int GENERATED ALWAYS AS (c1 * 2) STORED)
>
> We throw an error much earlier at
> logicalrep_rel_open->logicalrep_report_missing_attrs saying:
> ERROR: logical replication target relation "public.t1" is missing
> replicated column: "c2"
>
> 4) To simplify the code and reduce complexity, we can refactor the
> error checks to be included within the fetch_remote_table_info
> function. This way, the remotegenlist will not need to be prepared and
> passed to make_copy_attnamelist:
> + /*
> + * This loop checks for generated columns of the subscription table.
> + */
> + for (int i = 0; i < desc->natts; i++)
> {
> - attnamelist = lappend(attnamelist,
> -
> makeString(rel->remoterel.attnames[i]));
> + int remote_attnum;
> + Form_pg_attribute attr = TupleDescAttr(desc, i);
> +
> + if (!attr->attgenerated)
> + continue;
> +
> + remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
> +
> NameStr(attr->attname));
> +
> + if (remote_attnum >= 0)
> + {
> + /*
> + * Check if the subscription table generated
> column has same name
> + * as a non-generated column in the
> corresponding publication
> + * table.
> + */
> + if (!remotegenlist[remote_attnum])
> + ereport(ERROR,
> +
> (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("logical
> replication target relation \"%s.%s\" has a generated column \"%s\" "
> + "but
> corresponding column on source relation is not a generated column",
> +
> rel->remoterel.nspname, rel->remoterel.relname,
> NameStr(attr->attname))));
> +
> + /*
> + * 'localgenlist' records that this is a
> generated column in the
> + * subscription table. Later, we use this
> information to skip
> + * adding this column to the column list for COPY.
> + */
> + localgenlist[remote_attnum] = true;
> + }
> }
>
I have fixed all the comments and posted the v39 patches for them.
Please refer to the updated v39 Patches here in [1]. See [1] for the
changes added.
[1] https://www.postgresql.org/message-id/CAHv8RjLjb%2B98i5ZQUphivxdOZ3hSGLfq2SiWQetUvk8zGyAQwQ%40mail.g...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-09 09:23 ` vignesh C <[email protected]>
2024-10-10 00:45 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
5 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-09 09:23 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
>
> On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> >
> > Hi Shubham, here are my review comments for v36-0001.
> >
> > ======
> > 1. General - merge patches
> >
> > It is long past due when patches 0001 and 0002 should've been merged.
> > AFAIK the split was only because historically these parts had
> > different authors. But, keeping them separated is not helpful anymore.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 2.
> > Bitmapset *
> > -pub_collist_validate(Relation targetrel, List *columns)
> > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> >
> > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > so it should also be removed.
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 3.
> > /*
> > - * If the publication is FOR ALL TABLES then it is treated the same as
> > - * if there are no column lists (even if other publications have a
> > - * list).
> > + * To handle cases where the publish_generated_columns option isn't
> > + * specified for all tables in a publication, we must create a column
> > + * list that excludes generated columns. So, the publisher will not
> > + * replicate the generated columns.
> > */
> > - if (!pub->alltables)
> > + if (!(pub->alltables && pub->pubgencols))
> >
> > I still found that comment hard to understand. Does this mean to say
> > something like:
> >
> > ------
> > Process potential column lists for the following cases:
> >
> > a. Any publication that is not FOR ALL TABLES.
> >
> > b. When the publication is FOR ALL TABLES and
> > 'publish_generated_columns' is false.
> > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > so all columns will be replicated by default. However, if
> > 'publish_generated_columns' is set to false, column lists must still
> > be created to exclude any generated columns from being published
> > ------
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 4.
> > +SET client_min_messages = 'WARNING';
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> >
> > AFAIK you don't need to keep changing 'client_min_messages',
> > particularly now that you've removed the WARNING message that was
> > previously emitted.
> >
> > ~
> >
> > 5.
> > nit - minor comment changes.
> >
> > ======
> > Please refer to the attachment which implements any nits from above.
> >
>
> I have fixed all the given comments. Also, I have created a new 0003
> patch for the TAP-Tests related to the '011_generated.pl' file. I am
> planning to merge 0001 and 0003 patches once they will get fixed.
> The attached patches contain the required changes.
Few comments for v37-0002 patch:
1.a) We could include the output of each command execution like
"CREATE TABLE", "INSERT 0 3" and "CREATE PUBLICATION" as we have done
in other places like in [1]:
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS
AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
1.b) Similarly here too:
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS
AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub'
PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
1.c) Similarly here too:
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
1.d) Similarly here too:
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
1.e) Similarly here too:
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
2) All of the document changes of ddl.sgml, protocol.sgml,
create_publication.sgml can also be moved from 0001 patch to 0002
patch:
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb112..7b9c349343 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication
according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
3) I felt "(except generated columns)" should be removed from here too:
<variablelist>
<varlistentry id="protocol-logicalrep-message-formats-TupleData">
<term>TupleData</term>
<listitem>
<variablelist>
<varlistentry>
<term>Int16</term>
<listitem>
<para>
Number of columns.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
Next, one of the following submessages appears for each column
(except generated columns):
[1] - https://www.postgresql.org/docs/devel/logical-replication-subscription.html#LOGICAL-REPLICATION-SUBS...
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 09:23 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-10 00:45 ` Peter Smith <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-10 00:45 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi - Here is patch set v38 to address documentation review comments.
Changes from v37:
v38-0001.
- The code is unchanged.
- The SGML changes previously in 0001 are now in 0002.
- In passing, the commit message was improved to address review
comments from Peter [1].
v38-0002.
- Now includes all SGML changes previously in patch 0001.
- Addresses all docs review comments from Vignesh [2].
v38-0003.
- Unchanged.
======
[1] https://www.postgresql.org/message-id/CAHut%2BPv8Do3b7QhzHg7dRWhO317ZFZKY_mYQaFBOWVQ-P1805A%40mail.g...
[2] https://www.postgresql.org/message-id/CALDaNm3VcrDdF2A2x5QTPhajUxy150z3wsV4W4OGOwTFE4--Wg%40mail.gma...
Kind Regards,
Peter Smith.
Fujitsu Australia
//////////
Details are below:
On Wed, Oct 9, 2024 at 8:24 PM vignesh C <[email protected]> wrote:
>
> On Tue, 8 Oct 2024 at 11:37, Shubham Khanna <[email protected]> wrote:
> >
> > On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> > >
> > > Hi Shubham, here are my review comments for v36-0001.
> > >
> > > ======
> > > 1. General - merge patches
> > >
> > > It is long past due when patches 0001 and 0002 should've been merged.
> > > AFAIK the split was only because historically these parts had
> > > different authors. But, keeping them separated is not helpful anymore.
> > >
> > > ======
> > > src/backend/catalog/pg_publication.c
> > >
> > > 2.
> > > Bitmapset *
> > > -pub_collist_validate(Relation targetrel, List *columns)
> > > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> > >
> > > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > > so it should also be removed.
> > >
> > > ======
> > > src/backend/replication/pgoutput/pgoutput.c
> > >
> > > 3.
> > > /*
> > > - * If the publication is FOR ALL TABLES then it is treated the same as
> > > - * if there are no column lists (even if other publications have a
> > > - * list).
> > > + * To handle cases where the publish_generated_columns option isn't
> > > + * specified for all tables in a publication, we must create a column
> > > + * list that excludes generated columns. So, the publisher will not
> > > + * replicate the generated columns.
> > > */
> > > - if (!pub->alltables)
> > > + if (!(pub->alltables && pub->pubgencols))
> > >
> > > I still found that comment hard to understand. Does this mean to say
> > > something like:
> > >
> > > ------
> > > Process potential column lists for the following cases:
> > >
> > > a. Any publication that is not FOR ALL TABLES.
> > >
> > > b. When the publication is FOR ALL TABLES and
> > > 'publish_generated_columns' is false.
> > > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > > so all columns will be replicated by default. However, if
> > > 'publish_generated_columns' is set to false, column lists must still
> > > be created to exclude any generated columns from being published
> > > ------
> > >
> > > ======
> > > src/test/regress/sql/publication.sql
> > >
> > > 4.
> > > +SET client_min_messages = 'WARNING';
> > > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> > >
> > > AFAIK you don't need to keep changing 'client_min_messages',
> > > particularly now that you've removed the WARNING message that was
> > > previously emitted.
> > >
> > > ~
> > >
> > > 5.
> > > nit - minor comment changes.
> > >
> > > ======
> > > Please refer to the attachment which implements any nits from above.
> > >
> >
> > I have fixed all the given comments. Also, I have created a new 0003
> > patch for the TAP-Tests related to the '011_generated.pl' file. I am
> > planning to merge 0001 and 0003 patches once they will get fixed.
> > The attached patches contain the required changes.
>
> Few comments for v37-0002 patch:
> 1.a) We could include the output of each command execution like
> "CREATE TABLE", "INSERT 0 3" and "CREATE PUBLICATION" as we have done
> in other places like in [1]:
> +test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS
> AS (a + 1) STORED);
> +test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
> +test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
>
> 1.b) Similarly here too:
> +test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS
> AS (a * 100) STORED);
> +test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub'
> PUBLICATION pub1;
> +test_sub=# SELECT * from tab_gen_to_gen;
>
> 1.c) Similarly here too:
> +<programlisting>
> +test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
> +test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
> +test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
> +
> +test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
> +test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
> +test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
> +</programlisting>
> +<programlisting>
> +test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
> +test_sub-# c int,
> +test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
> +
> +test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
> +test_sub-# c int,
> +test_sub-# d int);
>
> 1.d) Similarly here too:
> +<programlisting>
> +test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
> +test_pub-# WITH (publish_generated_columns=false);
> +</programlisting>
> +<programlisting>
> +test_sub=# CREATE SUBSCRIPTION sub1
> +test_sub-# CONNECTION 'dbname=test_pub'
> +test_sub-# PUBLICATION pub1;
> +</programlisting>
>
> 1.e) Similarly here too:
> + Insert some data to the publisher tables:
> +<programlisting>
> +test_pub=# INSERT INTO t1 VALUES (1,2);
> +test_pub=# INSERT INTO t2 VALUES (1,2);
>
OK. All the above are fixed in v38-0002. I had thought this change was
mostly a waste of space, but I hadn't noticed there was a precedent.
IMO consistency is better, so I have made all the changes as you
suggested.
> 2) All of the document changes of ddl.sgml, protocol.sgml,
> create_publication.sgml can also be moved from 0001 patch to 0002
> patch:
> diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
> index 8ab0ddb112..7b9c349343 100644
> --- a/doc/src/sgml/ddl.sgml
> +++ b/doc/src/sgml/ddl.sgml
> @@ -514,8 +514,10 @@ CREATE TABLE people (
> </listitem>
> <listitem>
> <para>
> - Generated columns are skipped for logical replication and cannot be
> - specified in a <command>CREATE PUBLICATION</command> column list.
> + Generated columns may be skipped during logical replication
> according to the
> + <command>CREATE PUBLICATION</command> parameter
> + <link linkend="sql-createpublication-params-with-publish-generated-columns">
> + <literal>publish_generated_columns</literal></link>.
>
OK. SGML updates are moved to patch 0002 in v38.
> 3) I felt "(except generated columns)" should be removed from here too:
> <variablelist>
> <varlistentry id="protocol-logicalrep-message-formats-TupleData">
> <term>TupleData</term>
> <listitem>
> <variablelist>
> <varlistentry>
> <term>Int16</term>
> <listitem>
> <para>
> Number of columns.
> </para>
> </listitem>
> </varlistentry>
> </variablelist>
>
> <para>
> Next, one of the following submessages appears for each column
> (except generated columns):
>
OK. This is fixed in v38. This change is in "53.9. Logical Replication
Message Formats".
Attachments:
[application/octet-stream] v38-0002-DOCS-Generated-Column-Replication.patch (13.8K, ../../CAHut+PsPJfg5qEouz4BFP+Ja081Jxgkz4k_boSFb-_yP7z4qZw@mail.gmail.com/2-v38-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From e3f11bf9fce88251ea902d4b9f6cc8002a650bcd Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 10 Oct 2024 11:19:22 +1100
Subject: [PATCH v38] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 4 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 4 +-
doc/src/sgml/ref/create_publication.sgml | 16 ++
4 files changed, 310 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb..192180d 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0..7a8524e 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1405,6 +1405,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
</para>
<para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
+ <para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
is not supported.
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f..71b6b2a 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5de..c13cd4d 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,22 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
--
1.8.3.1
[application/octet-stream] v38-0003-Tap-tests-for-generated-columns.patch (14.1K, ../../CAHut+PsPJfg5qEouz4BFP+Ja081Jxgkz4k_boSFb-_yP7z4qZw@mail.gmail.com/3-v38-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 6dd2b803cbea462a974679be3f63585f514e9b5c Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v38] Tap tests for generated columns
Add tests for all combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna, Peter Smith
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 318 +++++++++++++++++++++++++++++++
1 file changed, 318 insertions(+)
mode change 100644 => 100755 src/test/subscription/t/011_generated.pl
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
old mode 100644
new mode 100755
index 8b2e5f4..5dd5965
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,322 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for all combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, all combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription with copy_data=false.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = false);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+# XXX copy_data=false for now. This will be changed later.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is($result, qq(),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# Test: no column list, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO gen_to_nogen VALUES (1, 1);
+ INSERT INTO gen_to_nogen2 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen(a, b, gen2), gen_to_nogen2 WITH (publish_generated_columns=false);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen2 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen2 ORDER BY c");
+is($result, qq(1|1||),
+ 'gen_to_nogen2 initial sync, when publish_generated_columns=false');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data excluding generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO nogen_to_gen VALUES (1, 1);
+ INSERT INTO nogen_to_gen2 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen, nogen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen2 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen ORDER BY a");
+is($result, qq(1|1||),
+ 'nogen_to_gen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen2 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO gen_to_nogen3 VALUES (1, 1);
+ INSERT INTO gen_to_nogen4 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table gen_to_nogen3(a, b, gen2), gen_to_nogen4 WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE gen_to_nogen3 (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE gen_to_nogen4 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen3 ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen4 ORDER BY c");
+is($result, qq(1|1|2|2),
+ 'gen_to_nogen4 initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher publishes only the data of the columns specified in
+# column list skipping other generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen3 (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int GENERATED ALWAYS AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO nogen_to_gen3 VALUES (1, 1);
+ INSERT INTO nogen_to_gen4 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen3, nogen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen3 (a int, b int, gen1 int, gen2 int);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen3 ORDER BY a");
+is($result, qq(1|1|2|2),
+ 'nogen_to_gen3 initial sync, when publish_generated_columns=true');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_gen4 ORDER BY c");
+is($result, qq(||2|),
+ 'nogen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Publisher replicates all columns if publish_generated_columns is
+# enabled and there is no column list
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_nogen (a int, b int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO nogen_to_nogen VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_nogen WITH (publish_generated_columns=true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_nogen (a int, b int, gen1 int, gen2 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM nogen_to_nogen ORDER BY a");
+is($result, qq(1|1|2|2),
+ 'nogen_to_nogen initial sync, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
1.8.3.1
[application/octet-stream] v38-0001-Enable-support-for-publish_generated_columns-opt.patch (96.1K, ../../CAHut+PsPJfg5qEouz4BFP+Ja081Jxgkz4k_boSFb-_yP7z4qZw@mail.gmail.com/4-v38-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 438a3228809b4179971406b472bfb48a4be03dd5 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 10 Oct 2024 09:13:10 +1100
Subject: [PATCH v38] Enable support for 'publish_generated_columns' option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
src/backend/catalog/pg_publication.c | 13 +-
src/backend/catalog/pg_subscription.c | 31 ++
src/backend/commands/publicationcmds.c | 33 +-
src/backend/commands/subscriptioncmds.c | 31 --
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 205 +++++++++---
src/backend/replication/pgoutput/pgoutput.c | 108 +++++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.in.c | 2 +-
src/include/catalog/pg_publication.h | 4 +
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 468 +++++++++++++++-------------
src/test/regress/sql/publication.sql | 40 ++-
src/test/subscription/t/031_column_list.pl | 4 +-
20 files changed, 670 insertions(+), 342 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2..7ebb851 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -511,7 +511,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +529,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +999,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1208,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc915..fcfbf86 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef3..0129db1 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc63..addf307 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -440,37 +440,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
-/*
* Check that the specified publications are present on the publisher.
*/
static void
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2..6b085e5 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b..338b083 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761..6f9e126 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,24 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +930,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +986,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1034,25 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1064,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -995,6 +1087,20 @@ fetch_remote_table_info(char *nspname, char *relname,
continue;
}
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the generated column option
+ * is specified nor it appears in the column list, we will skip it.
+ */
+ if (remotegenlist[natt] && !has_pub_with_pubgencols &&
+ !bms_is_member(attnum, included_cols))
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot, 2, &isnull));
Assert(!isnull);
@@ -1015,7 +1121,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1143,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1229,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1244,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1300,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1371,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024..d953a1a 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1009,6 +1009,37 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are excluded.
+ */
+static Bitmapset *
+prepare_nogen_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
+/*
* Initialize the column list.
*/
static void
@@ -1042,13 +1073,18 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * Process potential column lists for the following cases: a. Any
+ * publication that is not FOR ALL TABLES. b. When the publication is
+ * FOR ALL TABLES and 'publish_generated_columns' is false. FOR ALL
+ * TABLES publication doesn't have user-defined column lists, so all
+ * columns will be replicated by default. However, if
+ * 'publish_generated_columns' is set to false, column lists must
+ * still be created to exclude any generated columns from being
+ * published.
*/
- if (!pub->alltables)
+ if (!(pub->alltables && pub->pubgencols))
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = false;
/*
* Check for the presence of a column list in this publication.
@@ -1063,47 +1099,53 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (HeapTupleIsValid(cftuple))
{
+ bool pub_no_list = true;
+
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ pub_rel_has_collist = !pub_no_list;
+ }
- pgoutput_ensure_entry_cxt(data, entry);
+ /* Build the column list bitmap in the per-entry context. */
+ if (pub_rel_has_collist || !pub->pubgencols)
+ {
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ pgoutput_ensure_entry_cxt(data, entry);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_nogen_columns_bms(data, entry, desc);
- if (att->attisdropped || att->attgenerated)
- continue;
+ /* Get the number of live attributes. */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- nliveatts++;
- }
+ if (att->attisdropped)
+ continue;
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ nliveatts++;
}
- ReleaseSysCache(cftuple);
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
+ }
}
+
+ if (HeapTupleIsValid(cftuple))
+ ReleaseSysCache(cftuple);
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c38..1d79865 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed..c1552ea 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830..91a4c63 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91..16cbef3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index a9f4d20..5516b16 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a5..849b3a0 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec..158b444 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40..8cdb7af 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5..62e4820 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245e..e419ca8 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,44 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5..9724ba3 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,42 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- Remove generate columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5..2480aa4 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
1.8.3.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-09 16:48 ` Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
5 siblings, 1 reply; 72+ messages in thread
From: Masahiko Sawada @ 2024-10-09 16:48 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Peter Smith <[email protected]>; vignesh C <[email protected]>; Amit Kapila <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 7, 2024 at 11:07 PM Shubham Khanna
<[email protected]> wrote:
>
> On Fri, Oct 4, 2024 at 9:36 AM Peter Smith <[email protected]> wrote:
> >
> > Hi Shubham, here are my review comments for v36-0001.
> >
> > ======
> > 1. General - merge patches
> >
> > It is long past due when patches 0001 and 0002 should've been merged.
> > AFAIK the split was only because historically these parts had
> > different authors. But, keeping them separated is not helpful anymore.
> >
> > ======
> > src/backend/catalog/pg_publication.c
> >
> > 2.
> > Bitmapset *
> > -pub_collist_validate(Relation targetrel, List *columns)
> > +pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
> >
> > Since you removed the WARNING, this parameter 'pubgencols' is unused
> > so it should also be removed.
> >
> > ======
> > src/backend/replication/pgoutput/pgoutput.c
> >
> > 3.
> > /*
> > - * If the publication is FOR ALL TABLES then it is treated the same as
> > - * if there are no column lists (even if other publications have a
> > - * list).
> > + * To handle cases where the publish_generated_columns option isn't
> > + * specified for all tables in a publication, we must create a column
> > + * list that excludes generated columns. So, the publisher will not
> > + * replicate the generated columns.
> > */
> > - if (!pub->alltables)
> > + if (!(pub->alltables && pub->pubgencols))
> >
> > I still found that comment hard to understand. Does this mean to say
> > something like:
> >
> > ------
> > Process potential column lists for the following cases:
> >
> > a. Any publication that is not FOR ALL TABLES.
> >
> > b. When the publication is FOR ALL TABLES and
> > 'publish_generated_columns' is false.
> > A FOR ALL TABLES publication doesn't have user-defined column lists,
> > so all columns will be replicated by default. However, if
> > 'publish_generated_columns' is set to false, column lists must still
> > be created to exclude any generated columns from being published
> > ------
> >
> > ======
> > src/test/regress/sql/publication.sql
> >
> > 4.
> > +SET client_min_messages = 'WARNING';
> > +CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
> >
> > AFAIK you don't need to keep changing 'client_min_messages',
> > particularly now that you've removed the WARNING message that was
> > previously emitted.
> >
> > ~
> >
> > 5.
> > nit - minor comment changes.
> >
> > ======
> > Please refer to the attachment which implements any nits from above.
> >
>
> I have fixed all the given comments. Also, I have created a new 0003
> patch for the TAP-Tests related to the '011_generated.pl' file. I am
> planning to merge 0001 and 0003 patches once they will get fixed.
> The attached patches contain the required changes.
>
Regarding the 0001 patch, it seems to me that UPDATE and DELETE are
allowed on the table even if its replica identity is set to generated
columns that are not published. For example, consider the following
scenario:
create table t (a int not null, b int generated always as (a + 1)
stored not null);
create unique index t_idx on t (b);
alter table t replica identity using index t_idx;
create publication pub for table t with (publish_generated_columns = false);
insert into t values (1);
update t set a = 100 where a = 1;
The publication pub doesn't include the generated column 'b' which is
the replica identity of the table 't'. Therefore, the update message
generated by the last UPDATE would have NULL for the column 'b'. I
think we should not allow UPDATE and DELETE on such a table.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
@ 2024-10-22 10:50 ` Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 72+ messages in thread
From: Amit Kapila @ 2024-10-22 10:50 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 9, 2024 at 10:19 PM Masahiko Sawada <[email protected]> wrote:
>
> Regarding the 0001 patch, it seems to me that UPDATE and DELETE are
> allowed on the table even if its replica identity is set to generated
> columns that are not published. For example, consider the following
> scenario:
>
> create table t (a int not null, b int generated always as (a + 1)
> stored not null);
> create unique index t_idx on t (b);
> alter table t replica identity using index t_idx;
> create publication pub for table t with (publish_generated_columns = false);
> insert into t values (1);
> update t set a = 100 where a = 1;
>
> The publication pub doesn't include the generated column 'b' which is
> the replica identity of the table 't'. Therefore, the update message
> generated by the last UPDATE would have NULL for the column 'b'. I
> think we should not allow UPDATE and DELETE on such a table.
>
I see the same behavior even without a patch on the HEAD. See the
following example executed on HEAD:
postgres=# create table t (a int not null, b int generated always as (a + 1)
postgres(# stored not null);
CREATE TABLE
postgres=# create unique index t_idx on t (b);
CREATE INDEX
postgres=# alter table t replica identity using index t_idx;
ALTER TABLE
postgres=# create publication pub for table t;
CREATE PUBLICATION
postgres=# insert into t values (1);
INSERT 0 1
postgres=# update t set a = 100 where a = 1;
UPDATE 1
So, the update is allowed even when we don't publish generated
columns, if so, why do we need to handle it in this patch when the
user gave publish_generated_columns=false?
Also, on the subscriber side, I see the ERROR: "publisher did not send
replica identity column expected by the logical replication target
relation "public.t"".
Considering this, I feel if find this behavior buggy then we should
fix this separately rather than part of this patch. What do you think?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-22 16:11 ` Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
0 siblings, 1 reply; 72+ messages in thread
From: Masahiko Sawada @ 2024-10-22 16:11 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, Oct 22, 2024 at 3:50 AM Amit Kapila <[email protected]> wrote:
>
> On Wed, Oct 9, 2024 at 10:19 PM Masahiko Sawada <[email protected]> wrote:
> >
> > Regarding the 0001 patch, it seems to me that UPDATE and DELETE are
> > allowed on the table even if its replica identity is set to generated
> > columns that are not published. For example, consider the following
> > scenario:
> >
> > create table t (a int not null, b int generated always as (a + 1)
> > stored not null);
> > create unique index t_idx on t (b);
> > alter table t replica identity using index t_idx;
> > create publication pub for table t with (publish_generated_columns = false);
> > insert into t values (1);
> > update t set a = 100 where a = 1;
> >
> > The publication pub doesn't include the generated column 'b' which is
> > the replica identity of the table 't'. Therefore, the update message
> > generated by the last UPDATE would have NULL for the column 'b'. I
> > think we should not allow UPDATE and DELETE on such a table.
> >
>
> I see the same behavior even without a patch on the HEAD. See the
> following example executed on HEAD:
>
> postgres=# create table t (a int not null, b int generated always as (a + 1)
> postgres(# stored not null);
> CREATE TABLE
> postgres=# create unique index t_idx on t (b);
> CREATE INDEX
> postgres=# alter table t replica identity using index t_idx;
> ALTER TABLE
> postgres=# create publication pub for table t;
> CREATE PUBLICATION
> postgres=# insert into t values (1);
> INSERT 0 1
> postgres=# update t set a = 100 where a = 1;
> UPDATE 1
>
> So, the update is allowed even when we don't publish generated
> columns, if so, why do we need to handle it in this patch when the
> user gave publish_generated_columns=false?
>
> Also, on the subscriber side, I see the ERROR: "publisher did not send
> replica identity column expected by the logical replication target
> relation "public.t"".
Good point.
> Considering this, I feel if find this behavior buggy then we should
> fix this separately rather than part of this patch. What do you think?
Agreed. It's better to fix it separately.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
@ 2024-10-23 06:21 ` Amit Kapila <[email protected]>
2024-10-23 06:55 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 06:47 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
0 siblings, 3 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-23 06:21 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, Oct 22, 2024 at 9:42 PM Masahiko Sawada <[email protected]> wrote:
>
> On Tue, Oct 22, 2024 at 3:50 AM Amit Kapila <[email protected]> wrote:
> >
>
> > Considering this, I feel if find this behavior buggy then we should
> > fix this separately rather than part of this patch. What do you think?
>
> Agreed. It's better to fix it separately.
>
Thanks. One more thing that I didn't like about the patch is that it
used column_list to address the "publish_generated_columns = false"
case such that we build column_list without generated columns for the
same. The first problem is that it will add overhead to always probe
column_list during proto.c calls (for example during
logicalrep_write_attrs()), then it makes the column_list code complex
especially the handling in pgoutput_column_list_init(), and finally
this appears to be a misuse of column_list.
So, I suggest remembering this information in RelationSyncEntry and
then using it at the required places. We discussed above that
contradictory values of "publish_generated_columns" across
publications for the same relations are not accepted, so we can detect
that during get_rel_sync_entry() and give an ERROR for the same.
Additional comment on the 0003 patch
+# =============================================================================
+# Misc test.
+#
+# A "normal -> generated" replication.
+#
+# In this test case we use DROP EXPRESSION to change the subscriber generated
+# column into a normal column, then verify replication works ok.
+# =============================================================================
In patch 0003, why do we have the above test? This doesn't seem to be
directly related to this patch.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-23 06:55 ` Peter Smith <[email protected]>
2024-10-23 08:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-10-23 06:55 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 23, 2024 at 5:21 PM Amit Kapila <[email protected]> wrote:
> Additional comment on the 0003 patch
> +# =============================================================================
> +# Misc test.
> +#
> +# A "normal -> generated" replication.
> +#
> +# In this test case we use DROP EXPRESSION to change the subscriber generated
> +# column into a normal column, then verify replication works ok.
> +# =============================================================================
>
> In patch 0003, why do we have the above test? This doesn't seem to be
> directly related to this patch.
>
> --
Perhaps the test should be turned around, to test this feature more directly...
e.g. Replication of table tab(a int, b int) ==> tab(a int, b int, c int)
test_pub=# create table tab(a int, b int);
then, dynamically add a generated column "c" to the publisher table
test_pub=# alter table tab add column c int GENERATED ALWAYS AS (a + b) STORED;
test_pub=# insert into tab values (1,2);
then, verify that replication works for the newly added generated
column "c" to the existing normal column "c" at the subscriber.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-23 06:55 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-23 08:37 ` Amit Kapila <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-23 08:37 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 23, 2024 at 12:26 PM Peter Smith <[email protected]> wrote:
>
> On Wed, Oct 23, 2024 at 5:21 PM Amit Kapila <[email protected]> wrote:
>
> > Additional comment on the 0003 patch
> > +# =============================================================================
> > +# Misc test.
> > +#
> > +# A "normal -> generated" replication.
> > +#
> > +# In this test case we use DROP EXPRESSION to change the subscriber generated
> > +# column into a normal column, then verify replication works ok.
> > +# =============================================================================
> >
> > In patch 0003, why do we have the above test? This doesn't seem to be
> > directly related to this patch.
> >
> > --
>
> Perhaps the test should be turned around, to test this feature more directly...
>
> e.g. Replication of table tab(a int, b int) ==> tab(a int, b int, c int)
>
> test_pub=# create table tab(a int, b int);
>
> then, dynamically add a generated column "c" to the publisher table
> test_pub=# alter table tab add column c int GENERATED ALWAYS AS (a + b) STORED;
> test_pub=# insert into tab values (1,2);
>
> then, verify that replication works for the newly added generated
> column "c" to the existing normal column "c" at the subscriber.
>
This is testing whether the invalidation mechanism works for this case
which I see no reason to not work as this patch hasn't changed
anything in this regard. We should verify this but not sure it will
add much value in keeping this in regression tests.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-24 06:45 ` vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-24 06:45 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, 23 Oct 2024 at 11:51, Amit Kapila <[email protected]> wrote:
>
> On Tue, Oct 22, 2024 at 9:42 PM Masahiko Sawada <[email protected]> wrote:
> >
> > On Tue, Oct 22, 2024 at 3:50 AM Amit Kapila <[email protected]> wrote:
> > >
> >
> > > Considering this, I feel if find this behavior buggy then we should
> > > fix this separately rather than part of this patch. What do you think?
> >
> > Agreed. It's better to fix it separately.
> >
>
> Thanks. One more thing that I didn't like about the patch is that it
> used column_list to address the "publish_generated_columns = false"
> case such that we build column_list without generated columns for the
> same. The first problem is that it will add overhead to always probe
> column_list during proto.c calls (for example during
> logicalrep_write_attrs()), then it makes the column_list code complex
> especially the handling in pgoutput_column_list_init(), and finally
> this appears to be a misuse of column_list.
I simplified the process into three steps: a) Iterate through the
column list of publications and raise an error if there are any
discrepancies in the column list. b) Examine the non-column list of
publications to identify any conflicting options for
publish_generated_columns, and set this option accordingly. c)
Finally, verify that the columns in the column list align with the
table's columns based on the publish_generated_columns option.
> So, I suggest remembering this information in RelationSyncEntry and
> then using it at the required places. We discussed above that
> contradictory values of "publish_generated_columns" across
> publications for the same relations are not accepted, so we can detect
> that during get_rel_sync_entry() and give an ERROR for the same.
Resolved the issue by adding a new variable, pubgencols, to determine
whether the generated columns need to be published. This variable will
later be utilized in logicalrep_write_tuple and logicalrep_write_attrs
to replicate the generated columns to the subscriber if required.
> Additional comment on the 0003 patch
> +# =============================================================================
> +# Misc test.
> +#
> +# A "normal -> generated" replication.
> +#
> +# In this test case we use DROP EXPRESSION to change the subscriber generated
> +# column into a normal column, then verify replication works ok.
> +# =============================================================================
>
> In patch 0003, why do we have the above test? This doesn't seem to be
> directly related to this patch.
Removed this.
The attached v41 version patch has the changes for the same.
Regards,
Vignesh
Attachments:
[text/x-patch] v41-0003-Tap-tests-for-generated-columns.patch (11.1K, ../../CALDaNm1GCmzzGV6oGOB9g+LOA1j-8N_EFR3YNN1KaOr+oKdqJA@mail.gmail.com/2-v41-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 52280a1b2c2ac5abe8805dc372a064f50540ee29 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v41 3/3] Tap tests for generated columns
Add tests for the combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 275 +++++++++++++++++++++++
1 file changed, 275 insertions(+)
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index 8b2e5f4708..f02999812e 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,279 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for the combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, the combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+$node_subscriber->safe_psql('test_pgc_true', "DROP table tab_gen_to_nogen");
+$node_subscriber->safe_psql('postgres', "DROP DATABASE test_pgc_true");
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen, tab_gen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen2 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=false.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|),
+ 'tab_gen_to_gen initial sync, when publish_generated_columns=false');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen2 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|
+2|
+3|),
+ 'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
+);
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen2 incremental replication, when publish_generated_columns=false'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen3, tab_gen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen4 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2),
+ 'tab_gen_to_gen3 initial sync, when publish_generated_columns=true');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication.
+# Verify that column 'b' is replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen4 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2
+2|4
+3|6),
+ 'tab_gen_to_gen3 incremental replication, when publish_generated_columns=true'
+);
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen4 incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.34.1
[text/x-patch] v41-0002-DOCS-Generated-Column-Replication.patch (13.3K, ../../CALDaNm1GCmzzGV6oGOB9g+LOA1j-8N_EFR3YNN1KaOr+oKdqJA@mail.gmail.com/3-v41-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From c87cf06dee510e14f124f1660a78fbe609352db8 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 22 Oct 2024 20:19:49 +0530
Subject: [PATCH v41 2/3] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 4 +
4 files changed, 297 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 822d6c2a62..c383a4e0c8 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> option
- <link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>include_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..7a8524e825 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index e2895209a1..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.34.1
[text/x-patch] v41-0001-Enable-support-for-publish_generated_columns-opt.patch (111.8K, ../../CALDaNm1GCmzzGV6oGOB9g+LOA1j-8N_EFR3YNN1KaOr+oKdqJA@mail.gmail.com/4-v41-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From b40451cdaf30582a9cff69f917c129d87c398f09 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 8 Oct 2024 11:02:36 +0530
Subject: [PATCH v41 1/3] Enable support for 'publish_generated_columns'
option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 48 +-
src/backend/commands/publicationcmds.c | 33 +-
src/backend/replication/logical/proto.c | 71 +--
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 192 ++++++--
src/backend/replication/pgoutput/pgoutput.c | 194 ++++++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_publication.h | 5 +
src/include/replication/logicalproto.h | 19 +-
src/include/replication/logicalrelation.h | 3 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 508 ++++++++++++--------
src/test/regress/sql/publication.sql | 45 +-
src/test/subscription/t/031_column_list.pl | 4 +-
21 files changed, 851 insertions(+), 353 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index f6344b3b79..822d6c2a62 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>include_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..e2895209a1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7e5e357fd9..e937f8aed7 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -225,6 +225,41 @@ filter_partitions(List *table_infos)
}
}
+/*
+ * Returns true if the relation has column list associated with the
+ * publication, false if the relation has no column list associated with the
+ * publication.
+ */
+bool
+is_column_list_publication(Publication *pub, Oid relid)
+{
+ HeapTuple cftuple = NULL;
+ bool isnull = true;
+
+ if (pub->alltables)
+ return false;
+
+ cftuple = SearchSysCache2(PUBLICATIONRELMAP,
+ ObjectIdGetDatum(relid),
+ ObjectIdGetDatum(pub->oid));
+ if (HeapTupleIsValid(cftuple))
+ {
+ /* Lookup the column list attribute. */
+ (void) SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
+ Anum_pg_publication_rel_prattrs,
+ &isnull);
+ if (!isnull)
+ {
+ ReleaseSysCache(cftuple);
+ return true;
+ }
+
+ ReleaseSysCache(cftuple);
+ }
+
+ return false;
+}
+
/*
* Returns true if any schema is associated with the publication, false if no
* schema is associated with the publication.
@@ -500,8 +535,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
* valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * any unknown columns, system columns, or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
@@ -511,7 +545,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +563,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +1033,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1213,7 +1241,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped || (att->attgenerated && !pub->pubgencols))
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..0129db18c6 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..e62243e3b0 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -30,10 +30,11 @@
#define TRUNCATE_RESTART_SEQS (1<<1)
static void logicalrep_write_attrs(StringInfo out, Relation rel,
- Bitmapset *columns);
+ Bitmapset *columns, bool pubgencols);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
TupleTableSlot *slot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
static void logicalrep_read_attrs(StringInfo in, LogicalRepRelation *rel);
static void logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple);
@@ -52,6 +53,28 @@ column_in_column_list(int attnum, Bitmapset *columns)
return (columns == NULL || bms_is_member(attnum, columns));
}
+/*
+ * Check if the column should be published.
+ */
+static bool
+should_publish_column(Form_pg_attribute att, Bitmapset *columns,
+ bool pubgencols)
+{
+ if (att->attisdropped)
+ return false;
+
+ /*
+ * Skip publishing generated columns if the option is not specified or if
+ * they are not included in the column list.
+ */
+ if (att->attgenerated && !pubgencols && !columns)
+ return false;
+
+ if (!column_in_column_list(att->attnum, columns))
+ return false;
+
+ return true;
+}
/*
* Write BEGIN to the output stream.
@@ -412,7 +435,8 @@ logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn)
*/
void
logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns)
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_INSERT);
@@ -424,7 +448,7 @@ logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
pq_sendint32(out, RelationGetRelid(rel));
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -457,7 +481,7 @@ logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup)
void
logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, TupleTableSlot *newslot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_UPDATE);
@@ -478,11 +502,11 @@ logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, 'O'); /* old tuple follows */
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -532,7 +556,7 @@ logicalrep_read_update(StringInfo in, bool *has_oldtuple,
void
logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, bool binary,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
Assert(rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT ||
rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL ||
@@ -552,7 +576,7 @@ logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
/*
@@ -668,7 +692,7 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
*/
void
logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
char *relname;
@@ -690,7 +714,7 @@ logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, rel->rd_rel->relreplident);
/* send the attribute info */
- logicalrep_write_attrs(out, rel, columns);
+ logicalrep_write_attrs(out, rel, columns, pubgencols);
}
/*
@@ -767,7 +791,7 @@ logicalrep_read_typ(StringInfo in, LogicalRepTyp *ltyp)
*/
static void
logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
TupleDesc desc;
Datum *values;
@@ -781,10 +805,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -802,10 +823,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
if (isnull[i])
@@ -923,7 +941,8 @@ logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple)
* Write relation attribute metadata to the stream.
*/
static void
-logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
+logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns,
+ bool pubgencols)
{
TupleDesc desc;
int i;
@@ -938,10 +957,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -959,10 +975,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
/* REPLICA IDENTITY FULL means all columns are sent as part of key. */
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b01e..338b083696 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..49dc72e73b 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -692,21 +692,78 @@ process_syncing_tables(XLogRecPtr current_lsn)
}
/*
- * Create list of columns for COPY based on logical relation mapping.
+ * Create a list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
make_copy_attnamelist(LogicalRepRelMapEntry *rel)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip adding
+ * this column to the column list for COPY.
+ */
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!rel->remoterel.attremotegen[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct a column list for COPY, excluding columns that are
+ * subscription table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +848,20 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
- LogicalRepRelation *lrel, List **qual)
+fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool has_pub_with_pubgencols = false;
int natt;
ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,12 +904,13 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
@@ -873,8 +932,7 @@ fetch_remote_table_info(char *nspname, char *relname,
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
@@ -937,6 +995,43 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names.data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names.data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names.data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
pfree(pub_names.data);
}
@@ -948,20 +1043,22 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped"
" AND a.attrelid = %u"
- " ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
- lrel->remoteid);
+ " ORDER BY a.attnum", lrel->remoteid, lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1070,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ lrel->attremotegen = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -995,6 +1093,22 @@ fetch_remote_table_info(char *nspname, char *relname,
continue;
}
+ if (server_version >= 180000)
+ {
+ lrel->attremotegen[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
+ /*
+ * If the column is generated and neither the generated column
+ * option is specified nor it appears in the column list, we will
+ * skip it.
+ */
+ if (lrel->attremotegen[natt] && !has_pub_with_pubgencols && !included_cols)
+ {
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
rel_colname = TextDatumGetCString(slot_getattr(slot, 2, &isnull));
Assert(!isnull);
@@ -1037,7 +1151,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,6 +1237,7 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
@@ -1135,11 +1250,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (lrel.attremotegen[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1306,19 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * generated columns are published, because copy of generated columns
+ * is not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1376,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..c805f57a47 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -84,9 +84,6 @@ static bool publications_valid;
static List *LoadPublications(List *pubnames);
static void publication_invalidation_cb(Datum arg, int cacheid,
uint32 hashvalue);
-static void send_relation_and_attrs(Relation relation, TransactionId xid,
- LogicalDecodingContext *ctx,
- Bitmapset *columns);
static void send_repl_origin(LogicalDecodingContext *ctx,
RepOriginId origin_id, XLogRecPtr origin_lsn,
bool send_origin);
@@ -170,6 +167,9 @@ typedef struct RelationSyncEntry
*/
Bitmapset *columns;
+ /* Include publishing generated columns */
+ bool pubgencols;
+
/*
* Private context to store additional data for this entry - state for the
* row filter expressions, column list, etc.
@@ -213,6 +213,9 @@ static void init_rel_sync_cache(MemoryContext cachectx);
static void cleanup_rel_sync_cache(TransactionId xid, bool is_commit);
static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
Relation relation);
+static void send_relation_and_attrs(Relation relation, TransactionId xid,
+ LogicalDecodingContext *ctx,
+ RelationSyncEntry *relentry);
static void rel_sync_cache_relation_cb(Datum arg, Oid relid);
static void rel_sync_cache_publication_cb(Datum arg, int cacheid,
uint32 hashvalue);
@@ -731,11 +734,11 @@ maybe_send_schema(LogicalDecodingContext *ctx,
{
Relation ancestor = RelationIdGetRelation(relentry->publish_as_relid);
- send_relation_and_attrs(ancestor, xid, ctx, relentry->columns);
+ send_relation_and_attrs(ancestor, xid, ctx, relentry);
RelationClose(ancestor);
}
- send_relation_and_attrs(relation, xid, ctx, relentry->columns);
+ send_relation_and_attrs(relation, xid, ctx, relentry);
if (data->in_streaming)
set_schema_sent_in_streamed_txn(relentry, topxid);
@@ -749,9 +752,10 @@ maybe_send_schema(LogicalDecodingContext *ctx,
static void
send_relation_and_attrs(Relation relation, TransactionId xid,
LogicalDecodingContext *ctx,
- Bitmapset *columns)
+ RelationSyncEntry *relentry)
{
TupleDesc desc = RelationGetDescr(relation);
+ Bitmapset *columns = relentry->columns;
int i;
/*
@@ -766,12 +770,19 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
continue;
+ /*
+ * Skip publishing generated columns if the option is not specified or
+ * if they are not included in the column list.
+ */
+ if (att->attgenerated && !relentry->pubgencols && !columns)
+ continue;
+
/* Skip this attribute if it's not present in the column list */
if (columns != NULL && !bms_is_member(att->attnum, columns))
continue;
@@ -782,7 +793,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
}
OutputPluginPrepareWrite(ctx, false);
- logicalrep_write_rel(ctx->out, xid, relation, columns);
+ logicalrep_write_rel(ctx->out, xid, relation, columns, relentry->pubgencols);
OutputPluginWrite(ctx, false);
}
@@ -1008,6 +1019,112 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * Verify that the specified column list aligns with the columns selected for
+ * any non-column list publications (table/schema/all tables).
+ */
+static void
+check_conflicting_columns(List *publications, RelationSyncEntry *entry)
+{
+ Relation relation;
+ TupleDesc desc;
+ Bitmapset *cols = NULL;
+ ListCell *lc;
+
+ /* No column list specified or there is only one publication */
+ if (!entry->columns || (list_length(publications) == 1))
+ return;
+
+ relation = RelationIdGetRelation(entry->publish_as_relid);
+ desc = RelationGetDescr(relation);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attisdropped || (att->attgenerated && !entry->pubgencols))
+ continue;
+
+ cols = bms_add_member(cols, att->attnum);
+ }
+
+ foreach(lc, publications)
+ {
+ Publication *pub = lfirst(lc);
+
+ /* No need to check column list publications */
+ if (is_column_list_publication(pub, entry->publish_as_relid))
+ continue;
+
+ if (!bms_equal(entry->columns, cols))
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different column lists for table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
+ }
+
+ RelationClose(relation);
+}
+
+/*
+ * If the table contains a generated column, check for any conflicting
+ * values of publish_generated_columns in the publications.
+ */
+static void
+pgoutput_pubgencol_init(PGOutputData *data, List *publications,
+ RelationSyncEntry *entry)
+{
+ Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ TupleDesc desc = RelationGetDescr(relation);
+ bool gencolpresent = false;
+ ListCell *lc;
+ bool first = true;
+
+ /* Check if there is any generated column present */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (att->attgenerated)
+ {
+ gencolpresent = true;
+ break;
+ }
+ }
+
+ /* There is no generated columns to be published */
+ if (!gencolpresent)
+ {
+ entry->pubgencols = false;
+ return;
+ }
+
+ /*
+ * There may be a conflicting value for publish_generated_columns in the
+ * publications.
+ */
+ foreach(lc, publications)
+ {
+ Publication *pub = lfirst(lc);
+
+ /* No need to check column list publications */
+ if (is_column_list_publication(pub, entry->publish_as_relid))
+ continue;
+
+ if (first)
+ {
+ entry->pubgencols = pub->pubgencols;
+ first = false;
+ }
+ else if (entry->pubgencols != pub->pubgencols)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different values of publish_generated_columns for table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
+ }
+}
+
/*
* Initialize the column list.
*/
@@ -1071,52 +1188,27 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
/* Build the column list bitmap in the per-entry context. */
if (!pub_no_list) /* when not null */
{
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
-
pgoutput_ensure_entry_cxt(data, entry);
cols = pub_collist_to_bitmapset(cols, cfdatum,
entry->entry_cxt);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
+ if (first)
{
- Form_pg_attribute att = TupleDescAttr(desc, i);
-
- if (att->attisdropped || att->attgenerated)
- continue;
-
- nliveatts++;
- }
-
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
+ entry->columns = cols;
+ first = false;
}
+ else if (!bms_equal(entry->columns, cols))
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different column lists for table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
}
ReleaseSysCache(cftuple);
}
}
-
- if (first)
- {
- entry->columns = cols;
- first = false;
- }
- else if (!bms_equal(entry->columns, cols))
- ereport(ERROR,
- errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot use different column lists for table \"%s.%s\" in different publications",
- get_namespace_name(RelationGetNamespace(relation)),
- RelationGetRelationName(relation)));
} /* loop all subscribed publications */
RelationClose(relation);
@@ -1531,15 +1623,18 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
{
case REORDER_BUFFER_CHANGE_INSERT:
logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
logicalrep_write_update(ctx->out, xid, targetrel, old_slot,
- new_slot, data->binary, relentry->columns);
+ new_slot, data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_DELETE:
logicalrep_write_delete(ctx->out, xid, targetrel, old_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
default:
Assert(false);
@@ -2215,6 +2310,15 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
/* Initialize the column list */
pgoutput_column_list_init(data, rel_publications, entry);
+
+ /* Initialize publish generated columns value */
+ pgoutput_pubgencol_init(data, rel_publications, entry);
+
+ /*
+ * Check if there is conflict with the columns selected for the
+ * publication.
+ */
+ check_conflicting_columns(rel_publications, entry);
}
list_free(pubids);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e718..f9b38edd7e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6233,7 +6233,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6268,7 +6268,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6357,6 +6360,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6373,6 +6377,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6386,6 +6391,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6437,6 +6445,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6451,6 +6461,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6461,6 +6473,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 1be0056af7..fad2277991 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2261,7 +2261,7 @@ match_previous_words(int pattern_id,
"CURRENT_SCHEMA");
/* ALTER PUBLICATION <name> SET ( */
else if (Matches("ALTER", "PUBLICATION", MatchAny, MatchAnyN, "SET", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..2839847ba0 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -150,6 +154,7 @@ extern Oid GetTopMostAncestorInPublication(Oid puboid, List *ancestors,
extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
+extern bool is_column_list_publication(Publication *pub, Oid relid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index c409638a2e..f99b441f14 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -113,6 +113,7 @@ typedef struct LogicalRepRelation
char replident; /* replica identity */
char relkind; /* remote relation kind */
Bitmapset *attkeys; /* Bitmap of key columns */
+ bool *attremotegen; /* remote column is generated? */
} LogicalRepRelation;
/* Type mapping info */
@@ -223,20 +224,21 @@ extern void logicalrep_write_origin(StringInfo out, const char *origin,
XLogRecPtr origin_lsn);
extern char *logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn);
extern void logicalrep_write_insert(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *newslot,
- bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *newslot,
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup);
extern void logicalrep_write_update(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *oldslot,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *oldslot,
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols);
extern LogicalRepRelId logicalrep_read_update(StringInfo in,
bool *has_oldtuple, LogicalRepTupleData *oldtup,
LogicalRepTupleData *newtup);
extern void logicalrep_write_delete(StringInfo out, TransactionId xid,
Relation rel, TupleTableSlot *oldslot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
@@ -247,7 +249,8 @@ extern List *logicalrep_read_truncate(StringInfo in,
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
bool transactional, const char *prefix, Size sz, const char *message);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
- Relation rel, Bitmapset *columns);
+ Relation rel, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, TransactionId xid,
Oid typoid);
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40a56..8cdb7affbf 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..72943ef59a 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,84 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | t
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a)
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..1ee322fc4f 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,47 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.34.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-24 11:14 ` Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
0 siblings, 1 reply; 72+ messages in thread
From: Amit Kapila @ 2024-10-24 11:14 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, Oct 24, 2024 at 12:15 PM vignesh C <[email protected]> wrote:
>
> The attached v41 version patch has the changes for the same.
>
Please find comments for the new version as follows:
1.
+ Generated columns may be skipped during logical replication
according to the
+ <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>include_generated_columns</literal></link>.
The above statement doesn't sound to be clear. Can we change it to:
"Generated columns are allowed to be replicated during logical
replication according to the <command>CREATE PUBLICATION</command>
option .."?
2.
static void publication_invalidation_cb(Datum arg, int cacheid,
uint32 hashvalue);
-static void send_relation_and_attrs(Relation relation, TransactionId xid,
- LogicalDecodingContext *ctx,
- Bitmapset *columns);
static void send_repl_origin(LogicalDecodingContext *ctx,
...
...
static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
Relation relation);
+static void send_relation_and_attrs(Relation relation, TransactionId xid,
+ LogicalDecodingContext *ctx,
+ RelationSyncEntry *relentry);
Why the declaration of this function is changed?
3.
+ /*
+ * Skip publishing generated columns if the option is not specified or
+ * if they are not included in the column list.
+ */
+ if (att->attgenerated && !relentry->pubgencols && !columns)
In the comment above, shouldn't "specified or" be "specified and"?
4.
+pgoutput_pubgencol_init(PGOutputData *data, List *publications,
+ RelationSyncEntry *entry)
{
...
+ foreach(lc, publications)
+ {
+ Publication *pub = lfirst(lc);
+
+ /* No need to check column list publications */
+ if (is_column_list_publication(pub, entry->publish_as_relid))
Are we ignoring column_list publications because for such publications
the value of column_list prevails and we ignore
'publish_generated_columns' value? If so, it is not clear from the
comments.
5.
/* Initialize the column list */
pgoutput_column_list_init(data, rel_publications, entry);
+
+ /* Initialize publish generated columns value */
+ pgoutput_pubgencol_init(data, rel_publications, entry);
+
+ /*
+ * Check if there is conflict with the columns selected for the
+ * publication.
+ */
+ check_conflicting_columns(rel_publications, entry);
}
It looks odd to check conflicting column lists among publications
twice once in pgoutput_column_list_init() and then in
check_conflicting_columns(). Can we merge those?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-24 15:20 ` vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
0 siblings, 1 reply; 72+ messages in thread
From: vignesh C @ 2024-10-24 15:20 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, 24 Oct 2024 at 16:44, Amit Kapila <[email protected]> wrote:
>
> On Thu, Oct 24, 2024 at 12:15 PM vignesh C <[email protected]> wrote:
> >
> > The attached v41 version patch has the changes for the same.
> >
>
> Please find comments for the new version as follows:
> 1.
> + Generated columns may be skipped during logical replication
> according to the
> + <command>CREATE PUBLICATION</command> option
> + <link linkend="sql-createpublication-params-with-publish-generated-columns">
> + <literal>include_generated_columns</literal></link>.
>
> The above statement doesn't sound to be clear. Can we change it to:
> "Generated columns are allowed to be replicated during logical
> replication according to the <command>CREATE PUBLICATION</command>
> option .."?
Modified
> 2.
> static void publication_invalidation_cb(Datum arg, int cacheid,
> uint32 hashvalue);
> -static void send_relation_and_attrs(Relation relation, TransactionId xid,
> - LogicalDecodingContext *ctx,
> - Bitmapset *columns);
> static void send_repl_origin(LogicalDecodingContext *ctx,
> ...
> ...
> static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
> Relation relation);
> +static void send_relation_and_attrs(Relation relation, TransactionId xid,
> + LogicalDecodingContext *ctx,
> + RelationSyncEntry *relentry);
>
> Why the declaration of this function is changed?
Two changes were made: a) The function declaration need to be moved
down as the RelationSyncEntry structure is defined below. b) Bitmapset
was replaced with RelationSyncEntry to give send_relation_and_attrs
access to RelationSyncEntry.pubgencols and RelationSyncEntry.columns.
Instead of adding a new parameter to the function, RelationSyncEntry
was utilized, as it contains both pubgencols and columns members.
> 3.
> + /*
> + * Skip publishing generated columns if the option is not specified or
> + * if they are not included in the column list.
> + */
> + if (att->attgenerated && !relentry->pubgencols && !columns)
>
> In the comment above, shouldn't "specified or" be "specified and"?
Modified
> 4.
> +pgoutput_pubgencol_init(PGOutputData *data, List *publications,
> + RelationSyncEntry *entry)
>
> {
> ...
> + foreach(lc, publications)
> + {
> + Publication *pub = lfirst(lc);
> +
> + /* No need to check column list publications */
> + if (is_column_list_publication(pub, entry->publish_as_relid))
>
> Are we ignoring column_list publications because for such publications
> the value of column_list prevails and we ignore
> 'publish_generated_columns' value? If so, it is not clear from the
> comments.
Yes column takes precedence over publish_generated_columns value, so
column list publications are skipped. Modified the comments
accordingly.
> 5.
> /* Initialize the column list */
> pgoutput_column_list_init(data, rel_publications, entry);
> +
> + /* Initialize publish generated columns value */
> + pgoutput_pubgencol_init(data, rel_publications, entry);
> +
> + /*
> + * Check if there is conflict with the columns selected for the
> + * publication.
> + */
> + check_conflicting_columns(rel_publications, entry);
> }
>
> It looks odd to check conflicting column lists among publications
> twice once in pgoutput_column_list_init() and then in
> check_conflicting_columns(). Can we merge those?
Modified it to check from pgoutput_column_list_init
The v42 version patch attached at [1] has the changes for the same.
[1] - https://www.postgresql.org/message-id/CALDaNm2wFZRzSJLcNi_uMZcSUWuZ8%2Bkktc0n3Nfw9Fdti9WbVA%40mail.g...
Regards,
Vignesh
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
@ 2024-10-25 06:37 ` Amit Kapila <[email protected]>
2024-10-25 10:24 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
0 siblings, 2 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-25 06:37 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, Oct 24, 2024 at 8:50 PM vignesh C <[email protected]> wrote:
>
> The v42 version patch attached at [1] has the changes for the same.
>
Some more comments:
1.
@@ -1017,7 +1089,31 @@ pgoutput_column_list_init(PGOutputData *data,
List *publications,
{
ListCell *lc;
bool first = true;
+ Bitmapset *relcols = NULL;
Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ TupleDesc desc = RelationGetDescr(relation);
+ MemoryContext oldcxt = NULL;
+ bool collistpubexist = false;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ /*
+ * Prepare the columns that will be published for FOR ALL TABLES and
+ * FOR TABLES IN SCHEMA publication.
+ */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attisdropped || (att->attgenerated && !entry->pubgencols))
+ continue;
+
+ relcols = bms_add_member(relcols, att->attnum);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
This code is unnecessary for cases when the table's publication has a
column list. So, I suggest to form this list only when required. Also,
have an assertion that pubgencols value for entry and publication
matches.
2.
@@ -1115,10 +1186,17 @@ pgoutput_column_list_init(PGOutputData *data,
List *publications,
ereport(ERROR,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot use different column lists for table \"%s.%s\" in
different publications",
- get_namespace_name(RelationGetNamespace(relation)),
- RelationGetRelationName(relation)));
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
Is there a reason to make the above change? It appears to be a spurious change.
3.
+ /* Check if there is any generated column present */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (att->attgenerated)
Add one empty line between the above two lines.
4.
+ else if (entry->pubgencols != pub->pubgencols)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different values of publish_generated_columns for
table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
The last two lines are not aligned.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-25 10:24 ` Amit Kapila <[email protected]>
2024-10-25 14:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: Amit Kapila @ 2024-10-25 10:24 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 25, 2024 at 12:07 PM Amit Kapila <[email protected]> wrote:
>
> On Thu, Oct 24, 2024 at 8:50 PM vignesh C <[email protected]> wrote:
> >
> > The v42 version patch attached at [1] has the changes for the same.
> >
>
> Some more comments:
>
1.
+pgoutput_pubgencol_init(PGOutputData *data, List *publications,
+ RelationSyncEntry *entry)
Can we name it as check_and_init_gencol? I don't know if it is a good
idea to append a prefix pgoutput for local functions. It is primarily
used for exposed functions from pgoutput.c. I see that in a few cases
we do that for local functions as well but that is not a norm.
A related point:
+ /* Initialize publish generated columns value */
+ pgoutput_pubgencol_init(data, rel_publications, entry);
Accordingly change this comment to something like: "Check whether to
publish to generated columns.".
2.
+/*
+ * Returns true if the relation has column list associated with the
+ * publication, false if the relation has no column list associated with the
+ * publication.
+ */
+bool
+is_column_list_publication(Publication *pub, Oid relid)
...
...
How about naming the above function as has_column_list_defined()?
Also, you can write the above comment as: "Returns true if the
relation has column list associated with the publication, false
otherwise."
3.
+ /*
+ * The column list takes precedence over pubgencols, so skip checking
+ * column list publications.
+ */
+ if (is_column_list_publication(pub, entry->publish_as_relid))
Let's change this comment to: "The column list takes precedence over
publish_generated_columns option. Those will be checked later, see
pgoutput_column_list_init."
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 10:24 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-25 14:10 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-25 14:10 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 25, 2024 at 3:54 PM Amit Kapila <[email protected]> wrote:
>
> On Fri, Oct 25, 2024 at 12:07 PM Amit Kapila <[email protected]> wrote:
> >
> > On Thu, Oct 24, 2024 at 8:50 PM vignesh C <[email protected]> wrote:
> > >
> > > The v42 version patch attached at [1] has the changes for the same.
> > >
> >
> > Some more comments:
> >
>
> 1.
> +pgoutput_pubgencol_init(PGOutputData *data, List *publications,
> + RelationSyncEntry *entry)
>
> Can we name it as check_and_init_gencol? I don't know if it is a good
> idea to append a prefix pgoutput for local functions. It is primarily
> used for exposed functions from pgoutput.c. I see that in a few cases
> we do that for local functions as well but that is not a norm.
>
> A related point:
> + /* Initialize publish generated columns value */
> + pgoutput_pubgencol_init(data, rel_publications, entry);
>
> Accordingly change this comment to something like: "Check whether to
> publish to generated columns.".
>
Fixed.
> 2.
> +/*
> + * Returns true if the relation has column list associated with the
> + * publication, false if the relation has no column list associated with the
> + * publication.
> + */
> +bool
> +is_column_list_publication(Publication *pub, Oid relid)
> ...
> ...
>
> How about naming the above function as has_column_list_defined()?
> Also, you can write the above comment as: "Returns true if the
> relation has column list associated with the publication, false
> otherwise."
>
Fixed.
> 3.
> + /*
> + * The column list takes precedence over pubgencols, so skip checking
> + * column list publications.
> + */
> + if (is_column_list_publication(pub, entry->publish_as_relid))
>
> Let's change this comment to: "The column list takes precedence over
> publish_generated_columns option. Those will be checked later, see
> pgoutput_column_list_init."
>
Fixed.
The v43 version patch attached at [1] has the changes for the same.
[1] - https://www.postgresql.org/message-id/CAHv8RjJJJRzy83tG0nB90ivYcp7sFKTU%3D_BcQ-nUZ7VbHFwceA%40mail.g...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-25 14:04 ` Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 03:17 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
2024-10-28 06:54 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 3 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-25 14:04 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 25, 2024 at 12:07 PM Amit Kapila <[email protected]> wrote:
>
> On Thu, Oct 24, 2024 at 8:50 PM vignesh C <[email protected]> wrote:
> >
> > The v42 version patch attached at [1] has the changes for the same.
> >
>
> Some more comments:
> 1.
> @@ -1017,7 +1089,31 @@ pgoutput_column_list_init(PGOutputData *data,
> List *publications,
> {
> ListCell *lc;
> bool first = true;
> + Bitmapset *relcols = NULL;
> Relation relation = RelationIdGetRelation(entry->publish_as_relid);
> + TupleDesc desc = RelationGetDescr(relation);
> + MemoryContext oldcxt = NULL;
> + bool collistpubexist = false;
> +
> + pgoutput_ensure_entry_cxt(data, entry);
> +
> + oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
> +
> + /*
> + * Prepare the columns that will be published for FOR ALL TABLES and
> + * FOR TABLES IN SCHEMA publication.
> + */
> + for (int i = 0; i < desc->natts; i++)
> + {
> + Form_pg_attribute att = TupleDescAttr(desc, i);
> +
> + if (att->attisdropped || (att->attgenerated && !entry->pubgencols))
> + continue;
> +
> + relcols = bms_add_member(relcols, att->attnum);
> + }
> +
> + MemoryContextSwitchTo(oldcxt);
>
> This code is unnecessary for cases when the table's publication has a
> column list. So, I suggest to form this list only when required. Also,
> have an assertion that pubgencols value for entry and publication
> matches.
>
Modified and also included Assertion.
> 2.
> @@ -1115,10 +1186,17 @@ pgoutput_column_list_init(PGOutputData *data,
> List *publications,
> ereport(ERROR,
> errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> errmsg("cannot use different column lists for table \"%s.%s\" in
> different publications",
> - get_namespace_name(RelationGetNamespace(relation)),
> - RelationGetRelationName(relation)));
> + get_namespace_name(RelationGetNamespace(relation)),
> + RelationGetRelationName(relation)));
>
> Is there a reason to make the above change? It appears to be a spurious change.
>
Change is not required, removed now.
> 3.
> + /* Check if there is any generated column present */
> + for (int i = 0; i < desc->natts; i++)
> + {
> + Form_pg_attribute att = TupleDescAttr(desc, i);
> + if (att->attgenerated)
>
> Add one empty line between the above two lines.
>
Added.
> 4.
> + else if (entry->pubgencols != pub->pubgencols)
> + ereport(ERROR,
> + errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> + errmsg("cannot use different values of publish_generated_columns for
> table \"%s.%s\" in different publications",
> + get_namespace_name(RelationGetNamespace(relation)),
> + RelationGetRelationName(relation)));
>
> The last two lines are not aligned.
>
Modified.
The updated v43 version of patches contain the changes for the same.
Thanks and Regards,
Shubham Khanna.
Attachments:
[application/octet-stream] v43-0001-Support-logical-replication-of-generated-columns.patch (13.3K, ../../CAHv8RjJJJRzy83tG0nB90ivYcp7sFKTU=_BcQ-nUZ7VbHFwceA@mail.gmail.com/2-v43-0001-Support-logical-replication-of-generated-columns.patch)
download | inline diff:
From c83e216a05a3fe008eff32cd8c6d96ecee7a7832 Mon Sep 17 00:00:00 2001
From: Vignesh C <[email protected]>
Date: Fri, 25 Oct 2024 15:23:03 +0530
Subject: [PATCH v43 1/2] Support logical replication of generated columns in
column list.
Allow logical replication to publish generated columns if they are explicitly
mentioned in the column list.
---
doc/src/sgml/protocol.sgml | 4 +-
src/backend/catalog/pg_publication.c | 10 +----
src/backend/replication/logical/proto.c | 41 +++++++++++++--------
src/backend/replication/pgoutput/pgoutput.c | 26 ++++++++++---
src/test/regress/expected/publication.out | 21 +++++------
src/test/regress/sql/publication.sql | 2 +-
src/test/subscription/t/031_column_list.pl | 34 ++++++++++++++++-
7 files changed, 93 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7e5e357fd9..17a6093d06 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -500,8 +500,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
* valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * any unknown columns, system columns, or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
@@ -511,7 +510,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +528,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..cfc810a71a 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -52,6 +52,27 @@ column_in_column_list(int attnum, Bitmapset *columns)
return (columns == NULL || bms_is_member(attnum, columns));
}
+/*
+ * Check if the column should be published.
+ */
+static bool
+should_publish_column(Form_pg_attribute att, Bitmapset *columns)
+{
+ if (att->attisdropped)
+ return false;
+
+ /*
+ * Skip publishing generated columns if they are not included in the
+ * column list.
+ */
+ if (att->attgenerated && !columns)
+ return false;
+
+ if (!column_in_column_list(att->attnum, columns))
+ return false;
+
+ return true;
+}
/*
* Write BEGIN to the output stream.
@@ -781,10 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns))
continue;
nliveatts++;
@@ -802,10 +820,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns))
continue;
if (isnull[i])
@@ -938,10 +953,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns))
continue;
nliveatts++;
@@ -959,10 +971,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns))
continue;
/* REPLICA IDENTITY FULL means all columns are sent as part of key. */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..d59a8f5032 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,12 +766,19 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
continue;
+ /*
+ * Skip publishing generated columns if they are not included in the
+ * column list.
+ */
+ if (att->attgenerated && !columns)
+ continue;
+
/* Skip this attribute if it's not present in the column list */
if (columns != NULL && !bms_is_member(att->attnum, columns))
continue;
@@ -1074,6 +1081,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
int i;
int nliveatts = 0;
TupleDesc desc = RelationGetDescr(relation);
+ bool gencolpresent = false;
pgoutput_ensure_entry_cxt(data, entry);
@@ -1085,17 +1093,25 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
+ if (att->attgenerated)
+ {
+ if (bms_is_member(att->attnum, cols))
+ gencolpresent = true;
+
+ continue;
+ }
+
nliveatts++;
}
/*
- * If column list includes all the columns of the table,
- * set it to NULL.
+ * If column list includes all the columns of the table
+ * and there are no generated columns, set it to NULL.
*/
- if (bms_num_members(cols) == nliveatts)
+ if (bms_num_members(cols) == nliveatts && !gencolpresent)
{
bms_free(cols);
cols = NULL;
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..c248c2d717 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -687,43 +687,42 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
-ERROR: cannot use system column "ctid" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
ERROR: cannot use system column "ctid" in publication column list
-- error: duplicates not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, a);
-ERROR: duplicate column "a" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl5 (a, a);
ERROR: duplicate column "a" in publication column list
-- ok
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, c);
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
ALTER TABLE testpub_tbl5 DROP COLUMN c; -- no dice
-ERROR: cannot drop column c of table testpub_tbl5 because other objects depend on it
-DETAIL: publication of table testpub_tbl5 in publication testpub_fortable depends on column c of table testpub_tbl5
-HINT: Use DROP ... CASCADE to drop the dependent objects too.
-- ok: for insert-only publication, any column list is acceptable
ALTER PUBLICATION testpub_fortable_insert ADD TABLE testpub_tbl5 (b, c);
+ERROR: column "c" of relation "testpub_tbl5" does not exist
/* not all replica identities are good enough */
CREATE UNIQUE INDEX testpub_tbl5_b_key ON testpub_tbl5 (b, c);
+ERROR: column "c" does not exist
ALTER TABLE testpub_tbl5 ALTER b SET NOT NULL, ALTER c SET NOT NULL;
+ERROR: column "c" of relation "testpub_tbl5" does not exist
ALTER TABLE testpub_tbl5 REPLICA IDENTITY USING INDEX testpub_tbl5_b_key;
+ERROR: index "testpub_tbl5_b_key" for table "testpub_tbl5" does not exist
-- error: replica identity (b,c) is not covered by column list (a, c)
UPDATE testpub_tbl5 SET a = 1;
-ERROR: cannot update table "testpub_tbl5"
-DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: change the replica identity to "b", and column list to (a, c)
-- then update fails, because (a, c) does not cover replica identity
ALTER TABLE testpub_tbl5 REPLICA IDENTITY USING INDEX testpub_tbl5_b_key;
+ERROR: index "testpub_tbl5_b_key" for table "testpub_tbl5" does not exist
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, c);
+ERROR: column "c" of relation "testpub_tbl5" does not exist
UPDATE testpub_tbl5 SET a = 1;
-ERROR: cannot update table "testpub_tbl5"
-DETAIL: Column list used by the publication does not cover the replica identity.
/* But if upd/del are not published, it works OK */
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..9feb8442f2 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -413,7 +413,7 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..a0dc9bf3e6 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,7 +1202,7 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
# publication without any column (aka all columns as part of the columns
# list) are considered to have the same column list.
@@ -1275,6 +1275,38 @@ ok( $stderr =~
qr/cannot use different column lists for table "public.test_mix_1" in different publications/,
'different column lists detected');
+# TEST: Generated columns are considered for the column list.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE test_gen (a int PRIMARY KEY, b int GENERATED ALWAYS AS (a + 1) STORED);
+
+ CREATE PUBLICATION pub_gen FOR TABLE test_gen (a, b);
+));
+
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE test_gen (a int PRIMARY KEY, b int);
+));
+
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE SUBSCRIPTION sub_gen CONNECTION '$publisher_connstr' PUBLICATION pub_gen;
+));
+
+$node_subscriber->wait_for_subscription_sync;
+
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO test_gen VALUES (1);
+));
+
+$node_publisher->wait_for_catchup('sub_gen');
+
+is( $node_subscriber->safe_psql(
+ 'postgres', "SELECT * FROM test_gen ORDER BY a"),
+ qq(1|2),
+ 'replication with generated columns in column list');
+
# TEST: If the column list is changed after creating the subscription, we
# should catch the error reported by walsender.
--
2.41.0.windows.3
[application/octet-stream] v43-0003-DOCS-Generated-Column-Replication.patch (13.3K, ../../CAHv8RjJJJRzy83tG0nB90ivYcp7sFKTU=_BcQ-nUZ7VbHFwceA@mail.gmail.com/3-v43-0003-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From b2e11c2a3a944edfca6136d027c6aaea6d46318a Mon Sep 17 00:00:00 2001
From: Vignesh C <[email protected]>
Date: Thu, 24 Oct 2024 19:41:09 +0530
Subject: [PATCH v43 3/4] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 3 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 4 +
4 files changed, 297 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 577bcb4b71..a13f19bdbe 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -517,7 +517,8 @@ CREATE TABLE people (
Generated columns are allowed to be replicated during logical replication
according to the <command>CREATE PUBLICATION</command> option
<link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>include_generated_columns</literal></link>.
+ <literal>include_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..7a8524e825 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index e2895209a1..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.41.0.windows.3
[application/octet-stream] v43-0002-Enable-support-for-publish_generated_columns-opt.patch (106.6K, ../../CAHv8RjJJJRzy83tG0nB90ivYcp7sFKTU=_BcQ-nUZ7VbHFwceA@mail.gmail.com/4-v43-0002-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 5d27cafb4ecc0c1b3080623ccba4b3e531c99113 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Fri, 25 Oct 2024 17:16:37 +0530
Subject: [PATCH v43 2/2] Enable support for 'publish_generated_columns'
option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 71 ++-
src/backend/commands/publicationcmds.c | 33 +-
src/backend/replication/logical/proto.c | 46 +-
src/backend/replication/logical/tablesync.c | 60 ++-
src/backend/replication/pgoutput/pgoutput.c | 171 +++++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 17 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_publication.h | 7 +
src/include/replication/logicalproto.h | 18 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 523 ++++++++++++--------
src/test/regress/sql/publication.sql | 43 ++
18 files changed, 714 insertions(+), 337 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index f6344b3b79..577bcb4b71 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns are allowed to be replicated during logical replication
+ according to the <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>include_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 71b6b2a535..e2895209a1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column:
+ Next, one of the following submessages appears for each column (except generated columns):
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 17a6093d06..a662a453a9 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -225,6 +225,40 @@ filter_partitions(List *table_infos)
}
}
+/*
+ * Returns true if the relation has column list associated with the publication,
+ * false otherwise.
+ */
+bool
+has_column_list_defined(Publication *pub, Oid relid)
+{
+ HeapTuple cftuple = NULL;
+ bool isnull = true;
+
+ if (pub->alltables)
+ return false;
+
+ cftuple = SearchSysCache2(PUBLICATIONRELMAP,
+ ObjectIdGetDatum(relid),
+ ObjectIdGetDatum(pub->oid));
+ if (HeapTupleIsValid(cftuple))
+ {
+ /* Lookup the column list attribute. */
+ (void) SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
+ Anum_pg_publication_rel_prattrs,
+ &isnull);
+ if (!isnull)
+ {
+ ReleaseSysCache(cftuple);
+ return true;
+ }
+
+ ReleaseSysCache(cftuple);
+ }
+
+ return false;
+}
+
/*
* Returns true if any schema is associated with the publication, false if no
* schema is associated with the publication.
@@ -573,6 +607,40 @@ pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols, MemoryContext mcxt)
return result;
}
+/*
+ * Return a column list bitmap for the specified table.
+ *
+ * Generated columns are included if pubgencols is true.
+ *
+ * If mcxt isn't NULL, build the bitmapset in that context.
+ */
+Bitmapset *
+pub_getallcol_bitmapset(Relation relation, bool pubgencols,
+ MemoryContext mcxt)
+{
+ MemoryContext oldcxt = NULL;
+ Bitmapset *result = NULL;
+ TupleDesc desc = RelationGetDescr(relation);
+
+ if (mcxt)
+ oldcxt = MemoryContextSwitchTo(mcxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attisdropped || (att->attgenerated && !pubgencols))
+ continue;
+
+ result = bms_add_member(result, att->attnum);
+ }
+
+ if (mcxt)
+ MemoryContextSwitchTo(oldcxt);
+
+ return result;
+}
+
/*
* Insert new publication / schema mapping.
*/
@@ -998,6 +1066,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1205,7 +1274,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped || (att->attgenerated && !pub->pubgencols))
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..0129db18c6 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index cfc810a71a..a13225fd79 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -30,10 +30,11 @@
#define TRUNCATE_RESTART_SEQS (1<<1)
static void logicalrep_write_attrs(StringInfo out, Relation rel,
- Bitmapset *columns);
+ Bitmapset *columns, bool pubgencols);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
TupleTableSlot *slot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
static void logicalrep_read_attrs(StringInfo in, LogicalRepRelation *rel);
static void logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple);
@@ -56,16 +57,17 @@ column_in_column_list(int attnum, Bitmapset *columns)
* Check if the column should be published.
*/
static bool
-should_publish_column(Form_pg_attribute att, Bitmapset *columns)
+should_publish_column(Form_pg_attribute att, Bitmapset *columns,
+ bool pubgencols)
{
if (att->attisdropped)
return false;
/*
- * Skip publishing generated columns if they are not included in the
- * column list.
+ * Skip publishing generated columns if the option is not specified and if
+ * they are not included in the column list.
*/
- if (att->attgenerated && !columns)
+ if (att->attgenerated && !pubgencols && !columns)
return false;
if (!column_in_column_list(att->attnum, columns))
@@ -433,7 +435,8 @@ logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn)
*/
void
logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns)
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_INSERT);
@@ -445,7 +448,7 @@ logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
pq_sendint32(out, RelationGetRelid(rel));
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -478,7 +481,7 @@ logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup)
void
logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, TupleTableSlot *newslot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_UPDATE);
@@ -499,11 +502,11 @@ logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, 'O'); /* old tuple follows */
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -553,7 +556,7 @@ logicalrep_read_update(StringInfo in, bool *has_oldtuple,
void
logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, bool binary,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
Assert(rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT ||
rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL ||
@@ -573,7 +576,7 @@ logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
/*
@@ -689,7 +692,7 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
*/
void
logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
char *relname;
@@ -711,7 +714,7 @@ logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, rel->rd_rel->relreplident);
/* send the attribute info */
- logicalrep_write_attrs(out, rel, columns);
+ logicalrep_write_attrs(out, rel, columns, pubgencols);
}
/*
@@ -788,7 +791,7 @@ logicalrep_read_typ(StringInfo in, LogicalRepTyp *ltyp)
*/
static void
logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
TupleDesc desc;
Datum *values;
@@ -802,7 +805,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (!should_publish_column(att, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -820,7 +823,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (!should_publish_column(att, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
if (isnull[i])
@@ -938,7 +941,8 @@ logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple)
* Write relation attribute metadata to the stream.
*/
static void
-logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
+logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns,
+ bool pubgencols)
{
TupleDesc desc;
int i;
@@ -953,7 +957,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (!should_publish_column(att, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -971,7 +975,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (!should_publish_column(att, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
/* REPLICA IDENTITY FULL means all columns are sent as part of key. */
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index d4b5d210e3..92fb38840a 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -791,19 +791,20 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
- LogicalRepRelation *lrel, List **qual)
+fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
+ List **qual, bool *remotegencolpresent)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
int natt;
StringInfo pub_names = NULL;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -851,7 +852,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
@@ -868,8 +869,7 @@ fetch_remote_table_info(char *nspname, char *relname,
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
@@ -941,20 +941,21 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped"
" AND a.attrelid = %u"
- " ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
- lrel->remoteid);
+ " ORDER BY a.attnum", lrel->remoteid, lrel->remoteid);
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -998,6 +999,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 180000)
+ *remotegencolpresent |= DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1030,7 +1034,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
/* Reuse the already-built pub_names. */
Assert(pub_names != NULL);
@@ -1106,10 +1110,12 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &lrel, &qual,
+ &gencol_copy_needed);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1118,11 +1124,16 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1156,13 +1167,19 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * generated columns are published, because copy of generated columns
+ * is not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1220,7 +1237,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index d59a8f5032..b6babc2b95 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -84,9 +84,6 @@ static bool publications_valid;
static List *LoadPublications(List *pubnames);
static void publication_invalidation_cb(Datum arg, int cacheid,
uint32 hashvalue);
-static void send_relation_and_attrs(Relation relation, TransactionId xid,
- LogicalDecodingContext *ctx,
- Bitmapset *columns);
static void send_repl_origin(LogicalDecodingContext *ctx,
RepOriginId origin_id, XLogRecPtr origin_lsn,
bool send_origin);
@@ -170,6 +167,9 @@ typedef struct RelationSyncEntry
*/
Bitmapset *columns;
+ /* Include publishing generated columns */
+ bool pubgencols;
+
/*
* Private context to store additional data for this entry - state for the
* row filter expressions, column list, etc.
@@ -213,6 +213,9 @@ static void init_rel_sync_cache(MemoryContext cachectx);
static void cleanup_rel_sync_cache(TransactionId xid, bool is_commit);
static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
Relation relation);
+static void send_relation_and_attrs(Relation relation, TransactionId xid,
+ LogicalDecodingContext *ctx,
+ RelationSyncEntry *relentry);
static void rel_sync_cache_relation_cb(Datum arg, Oid relid);
static void rel_sync_cache_publication_cb(Datum arg, int cacheid,
uint32 hashvalue);
@@ -731,11 +734,11 @@ maybe_send_schema(LogicalDecodingContext *ctx,
{
Relation ancestor = RelationIdGetRelation(relentry->publish_as_relid);
- send_relation_and_attrs(ancestor, xid, ctx, relentry->columns);
+ send_relation_and_attrs(ancestor, xid, ctx, relentry);
RelationClose(ancestor);
}
- send_relation_and_attrs(relation, xid, ctx, relentry->columns);
+ send_relation_and_attrs(relation, xid, ctx, relentry);
if (data->in_streaming)
set_schema_sent_in_streamed_txn(relentry, topxid);
@@ -749,9 +752,10 @@ maybe_send_schema(LogicalDecodingContext *ctx,
static void
send_relation_and_attrs(Relation relation, TransactionId xid,
LogicalDecodingContext *ctx,
- Bitmapset *columns)
+ RelationSyncEntry *relentry)
{
TupleDesc desc = RelationGetDescr(relation);
+ Bitmapset *columns = relentry->columns;
int i;
/*
@@ -773,10 +777,10 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
continue;
/*
- * Skip publishing generated columns if they are not included in the
- * column list.
+ * Skip publishing generated columns if the option is not specified
+ * and if they are not included in the column list.
*/
- if (att->attgenerated && !columns)
+ if (att->attgenerated && !relentry->pubgencols && !columns)
continue;
/* Skip this attribute if it's not present in the column list */
@@ -789,7 +793,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
}
OutputPluginPrepareWrite(ctx, false);
- logicalrep_write_rel(ctx->out, xid, relation, columns);
+ logicalrep_write_rel(ctx->out, xid, relation, columns, relentry->pubgencols);
OutputPluginWrite(ctx, false);
}
@@ -1015,6 +1019,68 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * If the table contains a generated column, check for any conflicting
+ * values of publish_generated_columns in the publications.
+ */
+static void
+check_and_init_gencol(PGOutputData *data, List *publications,
+ RelationSyncEntry *entry)
+{
+ Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ TupleDesc desc = RelationGetDescr(relation);
+ bool gencolpresent = false;
+ ListCell *lc;
+ bool first = true;
+
+ /* Check if there is any generated column present */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attgenerated)
+ {
+ gencolpresent = true;
+ break;
+ }
+ }
+
+ /* There is no generated columns to be published */
+ if (!gencolpresent)
+ {
+ entry->pubgencols = false;
+ return;
+ }
+
+ /*
+ * There may be a conflicting value for publish_generated_columns in the
+ * publications.
+ */
+ foreach(lc, publications)
+ {
+ Publication *pub = lfirst(lc);
+
+ /*
+ * The column list takes precedence over publish_generated_columns option.
+ * Those will be checked later, see pgoutput_column_list_init.
+ */
+ if (has_column_list_defined(pub, entry->publish_as_relid))
+ continue;
+
+ if (first)
+ {
+ entry->pubgencols = pub->pubgencols;
+ first = false;
+ }
+ else if (entry->pubgencols != pub->pubgencols)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different values of publish_generated_columns for table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
+ }
+}
+
/*
* Initialize the column list.
*/
@@ -1025,6 +1091,8 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
ListCell *lc;
bool first = true;
Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ bool collistpubexist = false;
+ Bitmapset *relcols = NULL;
/*
* Find if there are any column lists for this relation. If there are,
@@ -1039,7 +1107,6 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
* need to check all the given publication-table mappings and report an
* error if any publications have a different column list.
*
- * FOR ALL TABLES and FOR TABLES IN SCHEMA imply "don't use column list".
*/
foreach(lc, publications)
{
@@ -1078,50 +1145,41 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
/* Build the column list bitmap in the per-entry context. */
if (!pub_no_list) /* when not null */
{
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
- bool gencolpresent = false;
-
+ collistpubexist = true;
pgoutput_ensure_entry_cxt(data, entry);
-
- cols = pub_collist_to_bitmapset(cols, cfdatum,
+ cols = pub_collist_to_bitmapset(NULL, cfdatum,
entry->entry_cxt);
-
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
-
- if (att->attisdropped)
- continue;
-
- if (att->attgenerated)
- {
- if (bms_is_member(att->attnum, cols))
- gencolpresent = true;
-
- continue;
- }
-
- nliveatts++;
- }
-
- /*
- * If column list includes all the columns of the table
- * and there are no generated columns, set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts && !gencolpresent)
- {
- bms_free(cols);
- cols = NULL;
- }
}
ReleaseSysCache(cftuple);
}
}
+ /*
+ * For non-column list publications—such as TABLE (without a column
+ * list), ALL TABLES, or ALL TABLES IN SCHEMA publications consider
+ * all columns of the table, including generated columns, based on the
+ * pubgencols option.
+ */
+ if (!cols)
+ {
+ Assert(pub->pubgencols == entry->pubgencols);
+
+ /*
+ * Retrieve the columns if they haven't been prepared yet, or if
+ * there are multiple publications.
+ */
+ if (!relcols && (list_length(publications) > 1))
+ {
+ pgoutput_ensure_entry_cxt(data, entry);
+ relcols = pub_getallcol_bitmapset(relation,
+ entry->pubgencols,
+ entry->entry_cxt);
+ }
+
+ cols = relcols;
+ }
+
if (first)
{
entry->columns = cols;
@@ -1135,6 +1193,13 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
RelationGetRelationName(relation)));
} /* loop all subscribed publications */
+ /*
+ * If no column list publications exit, columns will be selected later
+ * according to the generated columns option.
+ */
+ if (!collistpubexist)
+ entry->columns = NULL;
+
RelationClose(relation);
}
@@ -1547,15 +1612,18 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
{
case REORDER_BUFFER_CHANGE_INSERT:
logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
logicalrep_write_update(ctx->out, xid, targetrel, old_slot,
- new_slot, data->binary, relentry->columns);
+ new_slot, data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_DELETE:
logicalrep_write_delete(ctx->out, xid, targetrel, old_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
default:
Assert(false);
@@ -2229,6 +2297,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
/* Initialize the row filter */
pgoutput_row_filter_init(data, rel_publications, entry);
+ /* Check whether to publish to generated columns. */
+ check_and_init_gencol(data, rel_publications, entry);
+
/* Initialize the column list */
pgoutput_column_list_init(data, rel_publications, entry);
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e718..7d78fceed6 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6233,7 +6233,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6268,6 +6268,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6357,6 +6361,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6373,6 +6378,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6386,6 +6392,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6437,6 +6446,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6451,6 +6462,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6461,6 +6474,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 1be0056af7..fad2277991 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2261,7 +2261,7 @@ match_previous_words(int pattern_id,
"CURRENT_SCHEMA");
/* ALTER PUBLICATION <name> SET ( */
else if (Matches("ALTER", "PUBLICATION", MatchAny, MatchAnyN, "SET", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..bd68fa1f7c 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -150,6 +154,7 @@ extern Oid GetTopMostAncestorInPublication(Oid puboid, List *ancestors,
extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
+extern bool has_column_list_defined(Publication *pub, Oid relid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
@@ -158,5 +163,7 @@ extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
MemoryContext mcxt);
+extern Bitmapset *pub_getallcol_bitmapset(Relation relation, bool pubgencols,
+ MemoryContext mcxt);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index c409638a2e..fa6d66bff8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -223,20 +223,21 @@ extern void logicalrep_write_origin(StringInfo out, const char *origin,
XLogRecPtr origin_lsn);
extern char *logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn);
extern void logicalrep_write_insert(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *newslot,
- bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *newslot,
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup);
extern void logicalrep_write_update(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *oldslot,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *oldslot,
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols);
extern LogicalRepRelId logicalrep_read_update(StringInfo in,
bool *has_oldtuple, LogicalRepTupleData *oldtup,
LogicalRepTupleData *newtup);
extern void logicalrep_write_delete(StringInfo out, TransactionId xid,
Relation rel, TupleTableSlot *oldslot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
@@ -247,7 +248,8 @@ extern List *logicalrep_read_truncate(StringInfo in,
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
bool transactional, const char *prefix, Size sz, const char *message);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
- Relation rel, Bitmapset *columns);
+ Relation rel, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, TransactionId xid,
Oid typoid);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index c248c2d717..72943ef59a 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -689,50 +695,51 @@ DETAIL: Column list used by the publication does not cover the replica identity
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
-ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
+ERROR: cannot use system column "ctid" in publication column list
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
ERROR: cannot use system column "ctid" in publication column list
-- error: duplicates not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, a);
-ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
+ERROR: duplicate column "a" in publication column list
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl5 (a, a);
ERROR: duplicate column "a" in publication column list
-- ok
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, c);
-ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
ALTER TABLE testpub_tbl5 DROP COLUMN c; -- no dice
+ERROR: cannot drop column c of table testpub_tbl5 because other objects depend on it
+DETAIL: publication of table testpub_tbl5 in publication testpub_fortable depends on column c of table testpub_tbl5
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
-- ok: for insert-only publication, any column list is acceptable
ALTER PUBLICATION testpub_fortable_insert ADD TABLE testpub_tbl5 (b, c);
-ERROR: column "c" of relation "testpub_tbl5" does not exist
/* not all replica identities are good enough */
CREATE UNIQUE INDEX testpub_tbl5_b_key ON testpub_tbl5 (b, c);
-ERROR: column "c" does not exist
ALTER TABLE testpub_tbl5 ALTER b SET NOT NULL, ALTER c SET NOT NULL;
-ERROR: column "c" of relation "testpub_tbl5" does not exist
ALTER TABLE testpub_tbl5 REPLICA IDENTITY USING INDEX testpub_tbl5_b_key;
-ERROR: index "testpub_tbl5_b_key" for table "testpub_tbl5" does not exist
-- error: replica identity (b,c) is not covered by column list (a, c)
UPDATE testpub_tbl5 SET a = 1;
+ERROR: cannot update table "testpub_tbl5"
+DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: change the replica identity to "b", and column list to (a, c)
-- then update fails, because (a, c) does not cover replica identity
ALTER TABLE testpub_tbl5 REPLICA IDENTITY USING INDEX testpub_tbl5_b_key;
-ERROR: index "testpub_tbl5_b_key" for table "testpub_tbl5" does not exist
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, c);
-ERROR: column "c" of relation "testpub_tbl5" does not exist
UPDATE testpub_tbl5 SET a = 1;
+ERROR: cannot update table "testpub_tbl5"
+DETAIL: Column list used by the publication does not cover the replica identity.
/* But if upd/del are not published, it works OK */
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -916,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1124,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1165,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1246,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1259,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1288,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1314,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1385,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1396,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1417,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1429,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1441,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1452,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1463,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1474,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1505,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1517,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1599,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1620,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1748,6 +1755,84 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | t
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a)
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 9feb8442f2..1ee322fc4f 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -415,6 +417,7 @@ UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,47 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
--
2.41.0.windows.3
[application/octet-stream] v43-0004-Tap-tests-for-generated-columns.patch (10.6K, ../../CAHv8RjJJJRzy83tG0nB90ivYcp7sFKTU=_BcQ-nUZ7VbHFwceA@mail.gmail.com/5-v43-0004-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 054d6333592cff9ca6d4022d44aa960d85f762a8 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v43 4/4] Tap tests for generated columns
Add tests for the combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 270 +++++++++++++++++++++++
1 file changed, 270 insertions(+)
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index 8b2e5f4708..5ef7a9b4e6 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,274 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test case exercise logical replication where there is a
+# generated column on pub and a normal column on sub:
+# - generated -> normal
+#
+# Furthermore, the combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+$node_subscriber->safe_psql('test_pgc_true', "DROP table tab_gen_to_nogen");
+$node_subscriber->safe_psql('postgres', "DROP DATABASE test_pgc_true");
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab2 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab2, tab3(gen1) WITH (publish_generated_columns=false);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab2 (a) VALUES (1), (2);
+ INSERT INTO tab3 (a) VALUES (1), (2);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab2 (a int, gen1 int);
+ CREATE TABLE tab3 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=false.
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab2 ORDER BY a");
+is( $result, qq(1|
+2|),
+ 'tab2 initial sync, when publish_generated_columns=false');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab3 ORDER BY a");
+is( $result, qq(|2
+|4),
+ 'tab3 initial sync, when publish_generated_columns=false');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab2 VALUES (3), (4);
+ INSERT INTO tab3 VALUES (3), (4);
+));
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'gen1' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab2 ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|),
+ 'tab2 incremental replication, when publish_generated_columns=false');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab3 ORDER BY a");
+is( $result, qq(|2
+|4
+|6
+|8),
+ 'tab3 incremental replication, when publish_generated_columns=false');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab4 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab5 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab4, tab5(gen1) WITH (publish_generated_columns=true);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab4 (a) VALUES (1), (2);
+ INSERT INTO tab5 (a) VALUES (1), (2);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab4 (a int, gen1 int);
+ CREATE TABLE tab5 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=true.
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab4 ORDER BY a");
+is( $result, qq(1|2
+2|4),
+ 'tab4 initial sync, when publish_generated_columns=true');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab5 ORDER BY a");
+is( $result, qq(|2
+|4),
+ 'tab5 initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab4 VALUES (3), (4);
+ INSERT INTO tab5 VALUES (3), (4);
+));
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'gen1' is replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab4 ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8),
+ 'tab4 incremental replication, when publish_generated_columns=true');
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM tab5 ORDER BY a");
+is( $result, qq(|2
+|4
+|6
+|8),
+ 'tab5 incremental replication, when publish_generated_columns=true');
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.41.0.windows.3
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-28 02:13 ` Peter Smith <[email protected]>
2024-10-28 05:33 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-28 06:45 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2 siblings, 2 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-28 02:13 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Amit Kapila <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi, here are my review comments for patch v43-0001.
======
1. Missing docs update?
The CREATE PUBLICATION docs currently says:
When a column list is specified, only the named columns are
replicated. If no column list is specified, all columns of the table
are replicated through this publication, including any columns added
later.
~
For this patch, should that be updated to say "... all columns (except
generated columns) of the table are replicated..."
======
src/backend/replication/logical/proto.c
2.
+static bool
+should_publish_column(Form_pg_attribute att, Bitmapset *columns)
+{
+ if (att->attisdropped)
+ return false;
+
+ /*
+ * Skip publishing generated columns if they are not included in the
+ * column list.
+ */
+ if (att->attgenerated && !columns)
+ return false;
+
+ if (!column_in_column_list(att->attnum, columns))
+ return false;
+
+ return true;
+}
Here, I wanted to suggest that the whole "Skip publishing generated
columns" if-part is unnecessary because the next check
(!column_in_column_list) is going to return false for the same
scenario anyhow.
But, unfortunately, the "column_in_column_list" function has some
special NULL handling logic in it; this means none of this code is
quite what it seems to be (e.g. the function name
column_in_column_list is somewhat misleading)
IMO it would be better to change the column_in_column_list signature
-- add another boolean param to say if a NULL column list is allowed
or not. That will remove any subtle behaviour and then you can remove
the "if (att->attgenerated && !columns)" part.
======
src/backend/replication/pgoutput/pgoutput.c
3. send_relation_and_attrs
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
continue;
+ /*
+ * Skip publishing generated columns if they are not included in the
+ * column list.
+ */
+ if (att->attgenerated && !columns)
+ continue;
+
/* Skip this attribute if it's not present in the column list */
if (columns != NULL && !bms_is_member(att->attnum, columns))
continue;
~
Most of that code above looks to be doing the very same thing as the
new 'should_publish_column' in proto.c. Won't it be better to expose
the other function and share the common logic?
~~~
4. pgoutput_column_list_init
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
+ if (att->attgenerated)
+ {
+ if (bms_is_member(att->attnum, cols))
+ gencolpresent = true;
+
+ continue;
+ }
+
nliveatts++;
}
/*
- * If column list includes all the columns of the table,
- * set it to NULL.
+ * If column list includes all the columns of the table
+ * and there are no generated columns, set it to NULL.
*/
- if (bms_num_members(cols) == nliveatts)
+ if (bms_num_members(cols) == nliveatts && !gencolpresent)
{
Something seems not quite right (or maybe redundant) with this logic.
For example, because you unconditionally 'continue' for generated
columns, then AFAICT it is just not possible for bms_num_members(cols)
== nliveatts and at the same time 'gencolpresent' to be true. So you
could've just Asserted (!gencolpresent) instead of checking it in the
condition and mentioning it in the comment.
======
src/test/regress/expected/publication.out
5.
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
By allowing the above to work without giving ERROR, I think you've
broken many subsequent test expected results. e.g. I don't trust these
"expected" results anymore because I didn't think these next test
errors should have been affected, right?
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
-ERROR: cannot use system column "ctid" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication
"testpub_fortable"
Hmm - looks like a wrong expected result to me.
~
-- error: duplicates not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, a);
-ERROR: duplicate column "a" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication
"testpub_fortable"
Hmm - looks like a wrong expected result to me.
probably more like this...
======
src/test/subscription/t/031_column_list.pl
6.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE test_gen (a int PRIMARY KEY, b int);
+));
+
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE SUBSCRIPTION sub_gen CONNECTION '$publisher_connstr'
PUBLICATION pub_gen;
+));
Should combine these.
~~~
7.
+$node_publisher->wait_for_catchup('sub_gen');
+
+is( $node_subscriber->safe_psql(
+ 'postgres', "SELECT * FROM test_gen ORDER BY a"),
+ qq(1|2),
+ 'replication with generated columns in column list');
+
But, this is only testing normal replication. You should also include
some initial table data so you can test that the initial table
synchronization works too. Otherwise, I think current this patch has
no proof that the initial 'copy_data' even works at all.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-28 05:33 ` Amit Kapila <[email protected]>
2024-10-28 06:56 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 11:43 ` RE: Pgoutput not capturing the generated columns Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 2 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-28 05:33 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]> wrote:
>
> Hi, here are my review comments for patch v43-0001.
>
> ======
> src/backend/replication/logical/proto.c
>
> 2.
> +static bool
> +should_publish_column(Form_pg_attribute att, Bitmapset *columns)
> +{
> + if (att->attisdropped)
> + return false;
> +
> + /*
> + * Skip publishing generated columns if they are not included in the
> + * column list.
> + */
> + if (att->attgenerated && !columns)
> + return false;
> +
> + if (!column_in_column_list(att->attnum, columns))
> + return false;
> +
> + return true;
> +}
>
> Here, I wanted to suggest that the whole "Skip publishing generated
> columns" if-part is unnecessary because the next check
> (!column_in_column_list) is going to return false for the same
> scenario anyhow.
>
> But, unfortunately, the "column_in_column_list" function has some
> special NULL handling logic in it; this means none of this code is
> quite what it seems to be (e.g. the function name
> column_in_column_list is somewhat misleading)
>
> IMO it would be better to change the column_in_column_list signature
> -- add another boolean param to say if a NULL column list is allowed
> or not. That will remove any subtle behaviour and then you can remove
> the "if (att->attgenerated && !columns)" part.
>
The NULL column list still means all columns, so changing the behavior
as you are proposing doesn't make sense and would make the code
difficult to understand.
>
> 4. pgoutput_column_list_init
>
> - if (att->attisdropped || att->attgenerated)
> + if (att->attisdropped)
> continue;
>
> + if (att->attgenerated)
> + {
> + if (bms_is_member(att->attnum, cols))
> + gencolpresent = true;
> +
> + continue;
> + }
> +
> nliveatts++;
> }
>
> /*
> - * If column list includes all the columns of the table,
> - * set it to NULL.
> + * If column list includes all the columns of the table
> + * and there are no generated columns, set it to NULL.
> */
> - if (bms_num_members(cols) == nliveatts)
> + if (bms_num_members(cols) == nliveatts && !gencolpresent)
> {
>
> Something seems not quite right (or maybe redundant) with this logic.
> For example, because you unconditionally 'continue' for generated
> columns, then AFAICT it is just not possible for bms_num_members(cols)
> == nliveatts and at the same time 'gencolpresent' to be true. So you
> could've just Asserted (!gencolpresent) instead of checking it in the
> condition and mentioning it in the comment.
>
It seems part of the logic is redundant. I propose to change something
along the lines of the attached. I haven't tested the attached change
as it shows how we can improve this part of code.
--
With Regards,
Amit Kapila.
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index d59a8f5032..17aeb80637 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1081,7 +1081,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
int i;
int nliveatts = 0;
TupleDesc desc = RelationGetDescr(relation);
- bool gencolpresent = false;
+ bool att_gen_present = false;
pgoutput_ensure_entry_cxt(data, entry);
@@ -1098,20 +1098,19 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (att->attgenerated)
{
- if (bms_is_member(att->attnum, cols))
- gencolpresent = true;
-
- continue;
+ att_gen_present = true;
+ break;
}
nliveatts++;
}
/*
- * If column list includes all the columns of the table
- * and there are no generated columns, set it to NULL.
+ * Generated attributes are published only when they are
+ * present in the column list. Otherwise, a NULL column
+ * list means publish all columns.
*/
- if (bms_num_members(cols) == nliveatts && !gencolpresent)
+ if (!att_gen_present && bms_num_members(cols) == nliveatts)
{
bms_free(cols);
cols = NULL;
Attachments:
[text/plain] v43_0001_amit.1.patch.txt (1.3K, ../../CAA4eK1JjJcbkU2DeGTvPRB9XcU=4ftto63Z9M2r3sccu7K8XYQ@mail.gmail.com/2-v43_0001_amit.1.patch.txt)
download | inline diff:
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index d59a8f5032..17aeb80637 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1081,7 +1081,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
int i;
int nliveatts = 0;
TupleDesc desc = RelationGetDescr(relation);
- bool gencolpresent = false;
+ bool att_gen_present = false;
pgoutput_ensure_entry_cxt(data, entry);
@@ -1098,20 +1098,19 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (att->attgenerated)
{
- if (bms_is_member(att->attnum, cols))
- gencolpresent = true;
-
- continue;
+ att_gen_present = true;
+ break;
}
nliveatts++;
}
/*
- * If column list includes all the columns of the table
- * and there are no generated columns, set it to NULL.
+ * Generated attributes are published only when they are
+ * present in the column list. Otherwise, a NULL column
+ * list means publish all columns.
*/
- if (bms_num_members(cols) == nliveatts && !gencolpresent)
+ if (!att_gen_present && bms_num_members(cols) == nliveatts)
{
bms_free(cols);
cols = NULL;
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 05:33 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-28 06:56 ` Peter Smith <[email protected]>
2024-10-28 08:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-10-28 06:56 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 28, 2024 at 4:34 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]> wrote:
> >
> > Hi, here are my review comments for patch v43-0001.
> >
> > ======
> > src/backend/replication/logical/proto.c
> >
> > 2.
> > +static bool
> > +should_publish_column(Form_pg_attribute att, Bitmapset *columns)
> > +{
> > + if (att->attisdropped)
> > + return false;
> > +
> > + /*
> > + * Skip publishing generated columns if they are not included in the
> > + * column list.
> > + */
> > + if (att->attgenerated && !columns)
> > + return false;
> > +
> > + if (!column_in_column_list(att->attnum, columns))
> > + return false;
> > +
> > + return true;
> > +}
> >
> > Here, I wanted to suggest that the whole "Skip publishing generated
> > columns" if-part is unnecessary because the next check
> > (!column_in_column_list) is going to return false for the same
> > scenario anyhow.
> >
> > But, unfortunately, the "column_in_column_list" function has some
> > special NULL handling logic in it; this means none of this code is
> > quite what it seems to be (e.g. the function name
> > column_in_column_list is somewhat misleading)
> >
> > IMO it would be better to change the column_in_column_list signature
> > -- add another boolean param to say if a NULL column list is allowed
> > or not. That will remove any subtle behaviour and then you can remove
> > the "if (att->attgenerated && !columns)" part.
> >
>
> The NULL column list still means all columns, so changing the behavior
> as you are proposing doesn't make sense and would make the code
> difficult to understand.
>
My point was that the function 'column_in_column_list' would return
true even when there is no publication column list at all, so that
function name is misleading.
And, because in patch 0001 the generated columns only work when
specified via a column list it means now there is a difference
between:
- NULL (all columns specified in the column list) and
- NULL (no column list at all).
which seems strange and likely to cause confusion.
On closer inspection, this function 'column_in_column_list; is only
called from one place -- the new 'should_publish_column()'. I think
the function column_in_column_list should be thrown away and just
absorbed into the calling function 'should_publish_column'. Then the
misleading function name is eliminated, and the special NULL handling
can be commented on properly.
======
Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 05:33 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-28 06:56 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-28 08:50 ` Amit Kapila <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Amit Kapila @ 2024-10-28 08:50 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 28, 2024 at 12:27 PM Peter Smith <[email protected]> wrote:
>
> On Mon, Oct 28, 2024 at 4:34 PM Amit Kapila <[email protected]> wrote:
> >
> > On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]> wrote:
> > >
> > > Hi, here are my review comments for patch v43-0001.
> > >
> > > ======
> > > src/backend/replication/logical/proto.c
> > >
> > > 2.
> > > +static bool
> > > +should_publish_column(Form_pg_attribute att, Bitmapset *columns)
> > > +{
> > > + if (att->attisdropped)
> > > + return false;
> > > +
> > > + /*
> > > + * Skip publishing generated columns if they are not included in the
> > > + * column list.
> > > + */
> > > + if (att->attgenerated && !columns)
> > > + return false;
> > > +
> > > + if (!column_in_column_list(att->attnum, columns))
> > > + return false;
> > > +
> > > + return true;
> > > +}
> > >
> > > Here, I wanted to suggest that the whole "Skip publishing generated
> > > columns" if-part is unnecessary because the next check
> > > (!column_in_column_list) is going to return false for the same
> > > scenario anyhow.
> > >
> > > But, unfortunately, the "column_in_column_list" function has some
> > > special NULL handling logic in it; this means none of this code is
> > > quite what it seems to be (e.g. the function name
> > > column_in_column_list is somewhat misleading)
> > >
> > > IMO it would be better to change the column_in_column_list signature
> > > -- add another boolean param to say if a NULL column list is allowed
> > > or not. That will remove any subtle behaviour and then you can remove
> > > the "if (att->attgenerated && !columns)" part.
> > >
> >
> > The NULL column list still means all columns, so changing the behavior
> > as you are proposing doesn't make sense and would make the code
> > difficult to understand.
> >
>
> My point was that the function 'column_in_column_list' would return
> true even when there is no publication column list at all, so that
> function name is misleading.
>
> And, because in patch 0001 the generated columns only work when
> specified via a column list it means now there is a difference
> between:
> - NULL (all columns specified in the column list) and
> - NULL (no column list at all).
>
> which seems strange and likely to cause confusion.
>
This is no more strange than it was before the 0001 patch. Also, the
comment atop the function clarifies the special condition of the
function. OTOH, I am fine with pulling the check outside function as
you are proposing especially because now it is called from just one
place.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* RE: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 05:33 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-28 11:43 ` Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 0 replies; 72+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2024-10-28 11:43 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; Peter Smith <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
On Monday, October 28, 2024 1:34 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]>
> wrote:
>
> >
> > 4. pgoutput_column_list_init
> >
> > - if (att->attisdropped || att->attgenerated)
> > + if (att->attisdropped)
> > continue;
> >
> > + if (att->attgenerated)
> > + {
> > + if (bms_is_member(att->attnum, cols)) gencolpresent = true;
> > +
> > + continue;
> > + }
> > +
> > nliveatts++;
> > }
> >
> > /*
> > - * If column list includes all the columns of the table,
> > - * set it to NULL.
> > + * If column list includes all the columns of the table
> > + * and there are no generated columns, set it to NULL.
> > */
> > - if (bms_num_members(cols) == nliveatts)
> > + if (bms_num_members(cols) == nliveatts && !gencolpresent)
> > {
> >
> > Something seems not quite right (or maybe redundant) with this logic.
> > For example, because you unconditionally 'continue' for generated
> > columns, then AFAICT it is just not possible for bms_num_members(cols)
> > == nliveatts and at the same time 'gencolpresent' to be true. So you
> > could've just Asserted (!gencolpresent) instead of checking it in the
> > condition and mentioning it in the comment.
I think it's possible for the condition you mentioned to happen.
For example:
CREATE TABLE test_mix_4 (a int primary key, b int, d int GENERATED ALWAYS AS (a + 1) STORED);
CREATE PUBLICATION pub FOR TABLE test_mix_4(a, d);
> >
>
> It seems part of the logic is redundant. I propose to change something along the
> lines of the attached. I haven't tested the attached change as it shows how we
> can improve this part of code.
Thanks for the changes. I tried and faced an unexpected behavior
that the walsender will report Error "cannot use different column lists fo.."
in the following case:
Pub:
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int GENERATED ALWAYS AS (a + 1) STORED);
ALTER TABLE test_mix_4 DROP COLUMN c;
CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 (a, b);
CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4;
Sub:
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub_mix_7, pub_mix_8;
The pub_mix_7 publishes column a,b which should be converted to NULL
in pgoutput, but was not due to the check of att_gen_present.
Based on above, I feel we can keep the original code as it is.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-28 06:45 ` Amit Kapila <[email protected]>
2024-10-28 07:14 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: Amit Kapila @ 2024-10-28 06:45 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]> wrote:
>
> 7.
> +$node_publisher->wait_for_catchup('sub_gen');
> +
> +is( $node_subscriber->safe_psql(
> + 'postgres', "SELECT * FROM test_gen ORDER BY a"),
> + qq(1|2),
> + 'replication with generated columns in column list');
> +
>
> But, this is only testing normal replication. You should also include
> some initial table data so you can test that the initial table
> synchronization works too. Otherwise, I think current this patch has
> no proof that the initial 'copy_data' even works at all.
>
Per my tests, the initial copy doesn't work with 0001 alone. It needs
changes in table sync.c from the 0002 patch. Now, we can commit 0001
after fixing comments and mentioning in the commit message that this
patch supports only the replication of generated columns when
specified in the column list. The initial sync and replication of
generated columns when not specified in the column list will be
supported in future commits. OTOH, if the change to make table sync
work is simple, we can even combine that change.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 06:45 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-28 07:14 ` Peter Smith <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-28 07:14 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Shubham Khanna <[email protected]>; vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Oct 28, 2024 at 5:45 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Oct 28, 2024 at 7:43 AM Peter Smith <[email protected]> wrote:
> >
> > 7.
> > +$node_publisher->wait_for_catchup('sub_gen');
> > +
> > +is( $node_subscriber->safe_psql(
> > + 'postgres', "SELECT * FROM test_gen ORDER BY a"),
> > + qq(1|2),
> > + 'replication with generated columns in column list');
> > +
> >
> > But, this is only testing normal replication. You should also include
> > some initial table data so you can test that the initial table
> > synchronization works too. Otherwise, I think current this patch has
> > no proof that the initial 'copy_data' even works at all.
> >
>
> Per my tests, the initial copy doesn't work with 0001 alone. It needs
> changes in table sync.c from the 0002 patch. Now, we can commit 0001
> after fixing comments and mentioning in the commit message that this
> patch supports only the replication of generated columns when
> specified in the column list. The initial sync and replication of
> generated columns when not specified in the column list will be
> supported in future commits. OTOH, if the change to make table sync
> work is simple, we can even combine that change.
>
If this comes to a vote, then my vote is to refactor the necessary
tablesync COPY code back into patch 0001 so that patch 0001 can
replicate initial data properly stand alone.
Otherwise, (if we accept patch 0001 only partly works, like now) users
would have to jump through hoops to get any benefit from this patch by
itself. This is particularly true because the CREATE SUBSCRIPTION
'copy_data' parameter default is true, so patch 0001 is going to be
broken by default if there is any pre-existing table data when
publishing generated columns to default subscriptions.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* RE: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-28 03:17 ` Hayato Kuroda (Fujitsu) <[email protected]>
2024-10-28 13:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2024-10-28 03:17 UTC (permalink / raw)
To: 'Shubham Khanna' <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Amit Kapila <[email protected]>
Dear Shubham,
Thanks for updating the patch! I resumed reviewing the patch set.
Here are only cosmetic comments as my rehabilitation.
01. getPublications()
I feel we could follow the notation like getSubscriptions(), because number of
parameters became larger. How do you feel like attached?
02. fetch_remote_table_info()
```
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
```
I think no need to separate lines and add bracket. How about like below?
```
"SELECT DISTINCT gpt.attrs"
```
Best regards,
Hayato Kuroda
FUJITSU LIMITED
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1d79865058..5e2d28b447 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4292,30 +4292,26 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
+ appendPQExpBufferStr(query, "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "puballtables, p.pubinsert, p.pubupdate, p.pubdelete, ");
+
+ if (fout->remoteVersion >= 110000)
+ appendPQExpBufferStr(query, "p.pubtruncate, ");
+ else
+ appendPQExpBufferStr(query, "false AS pubtruncate, ");
+
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query, "p.pubviaroot, ");
+ else
+ appendPQExpBufferStr(query, "false AS pubviaroot, ");
+
if (fout->remoteVersion >= 180000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
- "FROM pg_publication p");
- else if (fout->remoteVersion >= 130000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
- else if (fout->remoteVersion >= 110000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
+ appendPQExpBufferStr(query, "p.pubgencols ");
else
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
+ appendPQExpBufferStr(query, "false AS pubgencols ");
+
+ appendPQExpBufferStr(query, "FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
Attachments:
[text/plain] refactor_dump.txt (2.1K, ../../TYAPR01MB56921B9E85AFD90631682A4DF54A2@TYAPR01MB5692.jpnprd01.prod.outlook.com/2-refactor_dump.txt)
download | inline diff:
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1d79865058..5e2d28b447 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4292,30 +4292,26 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
+ appendPQExpBufferStr(query, "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "puballtables, p.pubinsert, p.pubupdate, p.pubdelete, ");
+
+ if (fout->remoteVersion >= 110000)
+ appendPQExpBufferStr(query, "p.pubtruncate, ");
+ else
+ appendPQExpBufferStr(query, "false AS pubtruncate, ");
+
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query, "p.pubviaroot, ");
+ else
+ appendPQExpBufferStr(query, "false AS pubviaroot, ");
+
if (fout->remoteVersion >= 180000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
- "FROM pg_publication p");
- else if (fout->remoteVersion >= 130000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
- else if (fout->remoteVersion >= 110000)
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
+ appendPQExpBufferStr(query, "p.pubgencols ");
else
- appendPQExpBufferStr(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
- "FROM pg_publication p");
+ appendPQExpBufferStr(query, "false AS pubgencols ");
+
+ appendPQExpBufferStr(query, "FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 03:17 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
@ 2024-10-28 13:10 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-28 13:10 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Amit Kapila <[email protected]>
On Mon, Oct 28, 2024 at 8:47 AM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> Dear Shubham,
>
> Thanks for updating the patch! I resumed reviewing the patch set.
> Here are only cosmetic comments as my rehabilitation.
>
> 01. getPublications()
>
> I feel we could follow the notation like getSubscriptions(), because number of
> parameters became larger. How do you feel like attached?
>
I will handle this comment in a later set of patches.
> 02. fetch_remote_table_info()
>
> ```
> "SELECT DISTINCT"
> - " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
> - " THEN NULL ELSE gpt.attrs END)"
> + " (gpt.attrs)"
> ```
>
> I think no need to separate lines and add bracket. How about like below?
>
> ```
> "SELECT DISTINCT gpt.attrs"
> ```
>
Fixed this.
The v44 version patches attached at [1] have the changes for the same.
[1] - https://www.postgresql.org/message-id/CAHv8RjLvr8ZxX-1TcaxrZns1nwgrVUTO_2jhDdOPys0WgrDyKQ%40mail.gma...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* RE: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-28 06:54 ` Hayato Kuroda (Fujitsu) <[email protected]>
2024-10-28 11:45 ` RE: Pgoutput not capturing the generated columns Zhijie Hou (Fujitsu) <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2024-10-28 06:54 UTC (permalink / raw)
To: 'Shubham Khanna' <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Amit Kapila <[email protected]>
Dear Shubham,
More comments for v43-0001.
01. publication.out and publication.sql
I think your fix is not sufficient, even if it pass tests.
```
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
-ERROR: cannot use system column "ctid" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
ERROR: cannot use system column "ctid" in publication column list
-- error: duplicates not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, a);
-ERROR: duplicate column "a" in publication column list
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable
```
The error message is not match with the comment. The comment said that the table
has already been added in the publication. I think the first line [1] succeeded by your change
and testpub_tbl5 became a member at that time then upcoming ALTER statements failed
by the duplicate registration.
```
-- ok
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, c);
+ERROR: relation "testpub_tbl5" is already member of publication "testpub_fortable"
```
You said OK but same error happened.
```
ALTER TABLE testpub_tbl5 DROP COLUMN c; -- no dice
-ERROR: cannot drop column c of table testpub_tbl5 because other objects depend on it
-DETAIL: publication of table testpub_tbl5 in publication testpub_fortable depends on column c of table testpub_tbl5
-HINT: Use DROP ... CASCADE to drop the dependent objects too.
```
This statement should be failed because c was included in the column.
However, it succeeded because previous ALTER PUBLICATION was failed.
Upcoming SQLs wrongly thawed ERRORs because of this.
Please look at all of differences before doing copy-and-paste.
02. 031_column_list.pl
```
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
# publication without any column (aka all columns as part of the columns
# list) are considered to have the same column list.
```
Based on the comment, this case does not test the behavior of generated columns
anymore. So, I felt column 'd' could be removed from the case.
03. 031_column_list.pl
Can we test that generated columns won't be replaced if it does not included in
the column list?
[1]:
```
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
```
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 72+ messages in thread
* RE: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 06:54 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
@ 2024-10-28 11:45 ` Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2024-10-28 11:45 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; 'Shubham Khanna' <[email protected]>; +Cc: vignesh C <[email protected]>; Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Amit Kapila <[email protected]>
On Monday, October 28, 2024 2:54 PM Hayato Kuroda (Fujitsu) <[email protected]> wrote:
>
>
> 02. 031_column_list.pl
>
> ```
> -# TEST: Generated and dropped columns are not considered for the column
> list.
> +# TEST: Dropped columns are not considered for the column list.
> # So, the publication having a column list except for those columns and a #
> publication without any column (aka all columns as part of the columns #
> list) are considered to have the same column list.
> ```
>
> Based on the comment, this case does not test the behavior of generated
> columns anymore. So, I felt column 'd' could be removed from the case.
I think keeping the generated column can test the cases you mentioned
in comment #03, so we can modify the comments here to make that clear.
>
> 03. 031_column_list.pl
>
> Can we test that generated columns won't be replaced if it does not included in
> the column list?
As stated above, it can be covered in existing tests.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-24 06:47 ` Amit Kapila <[email protected]>
2024-10-24 15:17 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Amit Kapila @ 2024-10-24 06:47 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; vignesh C <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Oct 23, 2024 at 11:51 AM Amit Kapila <[email protected]> wrote:
>
> Thanks. One more thing that I didn't like about the patch is that it
> used column_list to address the "publish_generated_columns = false"
> case such that we build column_list without generated columns for the
> same. The first problem is that it will add overhead to always probe
> column_list during proto.c calls (for example during
> logicalrep_write_attrs()), then it makes the column_list code complex
> especially the handling in pgoutput_column_list_init(), and finally
> this appears to be a misuse of column_list.
>
> So, I suggest remembering this information in RelationSyncEntry and
> then using it at the required places. We discussed above that
> contradictory values of "publish_generated_columns" across
> publications for the same relations are not accepted, so we can detect
> that during get_rel_sync_entry() and give an ERROR for the same.
>
The changes in tablesync look complicated and I am not sure whether it
handles the conflicting publish_generated_columns option correctly. I
have few thoughts for the same.
* The handling of contradictory options in multiple publications needs
to be the same as for column lists. I think it is handled (a) during
subscription creation, (b) during copy in fetch_remote_table_info(),
and (c) during replication. See Peter's email
(https://www.postgresql.org/message-id/CAHut%2BPs985rc95cB2x5yMF56p6Lf192AmCJOpAtK_%2BC5YGUF2A%40mail...)
to understand why this new option has to be handled in the same way as
the column list.
* While fetching column list via pg_get_publication_tables(), we
should detect contradictory publish_generated_columns options similar
to column lists, and then after we get publish_generated_columns as
return value, we can even use that while fetching attribute
information.
A few additional comments:
1.
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (lrel.attremotegen[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
Can't we get this information from fetch_remote_table_info() instead
of traversing the entire column list again?
2.
@@ -1015,7 +1110,6 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
walrcv_clear_result(res);
Spurious line removal.
3. Why do we have to specifically exclude generated columns of a
subscriber-side table in make_copy_attnamelist()? Can't we rely on
fetch_remote_table_info() and logicalrep_rel_open() that the final
remote attrlist will contain the generated column only if the
subscriber doesn't have a generated column otherwise it would have
given an error in logicalrep_rel_open()?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:47 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
@ 2024-10-24 15:17 ` vignesh C <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: vignesh C @ 2024-10-24 15:17 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Shubham Khanna <[email protected]>; Peter Smith <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, 24 Oct 2024 at 12:17, Amit Kapila <[email protected]> wrote:
>
> On Wed, Oct 23, 2024 at 11:51 AM Amit Kapila <[email protected]> wrote:
> >
> > Thanks. One more thing that I didn't like about the patch is that it
> > used column_list to address the "publish_generated_columns = false"
> > case such that we build column_list without generated columns for the
> > same. The first problem is that it will add overhead to always probe
> > column_list during proto.c calls (for example during
> > logicalrep_write_attrs()), then it makes the column_list code complex
> > especially the handling in pgoutput_column_list_init(), and finally
> > this appears to be a misuse of column_list.
> >
> > So, I suggest remembering this information in RelationSyncEntry and
> > then using it at the required places. We discussed above that
> > contradictory values of "publish_generated_columns" across
> > publications for the same relations are not accepted, so we can detect
> > that during get_rel_sync_entry() and give an ERROR for the same.
> >
>
> The changes in tablesync look complicated and I am not sure whether it
> handles the conflicting publish_generated_columns option correctly. I
> have few thoughts for the same.
> * The handling of contradictory options in multiple publications needs
> to be the same as for column lists. I think it is handled (a) during
> subscription creation, (b) during copy in fetch_remote_table_info(),
> and (c) during replication. See Peter's email
> (https://www.postgresql.org/message-id/CAHut%2BPs985rc95cB2x5yMF56p6Lf192AmCJOpAtK_%2BC5YGUF2A%40mail...)
> to understand why this new option has to be handled in the same way as
> the column list.
>
> * While fetching column list via pg_get_publication_tables(), we
> should detect contradictory publish_generated_columns options similar
> to column lists, and then after we get publish_generated_columns as
> return value, we can even use that while fetching attribute
> information.
Modified it to detect conflicting column lists for publications,
allowing the detection of publish_generated_columns conflicts using
the same code currently used for column list detection.
> A few additional comments:
> 1.
> - /* Regular table with no row filter */
> - if (lrel.relkind == RELKIND_RELATION && qual == NIL)
> + /*
> + * Check if the remote table has any generated columns that should be
> + * copied.
> + */
> + for (int i = 0; i < relmapentry->remoterel.natts; i++)
> + {
> + if (lrel.attremotegen[i])
> + {
> + gencol_copy_needed = true;
> + break;
> + }
> + }
>
> Can't we get this information from fetch_remote_table_info() instead
> of traversing the entire column list again?
Yes, modified
> 2.
> @@ -1015,7 +1110,6 @@ fetch_remote_table_info(char *nspname, char *relname,
> ExecDropSingleTupleTableSlot(slot);
>
> lrel->natts = natt;
> -
> walrcv_clear_result(res);
>
> Spurious line removal.
Fixed this
> 3. Why do we have to specifically exclude generated columns of a
> subscriber-side table in make_copy_attnamelist()? Can't we rely on
> fetch_remote_table_info() and logicalrep_rel_open() that the final
> remote attrlist will contain the generated column only if the
> subscriber doesn't have a generated column otherwise it would have
> given an error in logicalrep_rel_open()?
Yes, it can be used. The updated v42 version of patch has the changes
for the same.
Regards,
Vignesh
Attachments:
[text/x-patch] v42-0001-Enable-support-for-publish_generated_columns-opt.patch (105.7K, ../../CALDaNm2wFZRzSJLcNi_uMZcSUWuZ8+kktc0n3Nfw9Fdti9WbVA@mail.gmail.com/2-v42-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 2c49f076e35ed82035784a696ce8e472d041c857 Mon Sep 17 00:00:00 2001
From: Khanna <[email protected]>
Date: Tue, 8 Oct 2024 11:02:36 +0530
Subject: [PATCH v42 1/3] Enable support for 'publish_generated_columns'
option.
Generated column values are not currently replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This patch supports the transmission of generated column information and data
alongside regular table changes. This behaviour is partly controlled by a new
publication parameter 'publish_generated_columns'.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
Generated columns can also be published if they are specified in a
publication column list. This overrides the parameter, so it works even if
'publish_generated_columns' is false.
When the subscription parameter 'copy_data' is true, then data is copied
during the initial table synchronization using the COPY command. The
normal COPY command does not copy generated columns, so if generated columns are
published we need to use a different form of the copy syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
~~
Behavior Summary:
A. when generated columns are published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column value is copied.
* Publisher generated column => subscriber generated column:
This will give ERROR.
B. when generated columns are not published
* Publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* Publisher not-generated column => subscriber generated column:
This will give ERROR.
* Publisher generated column => subscriber not-generated column:
The publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* Publisher generated column => subscriber generated column:
The publisher generated column is not replicated. The subscriber
generated column will be filled with the subscriber-side computed or
default data.
~
There is a change in 'pg_publication' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 48 +-
src/backend/commands/publicationcmds.c | 33 +-
src/backend/replication/logical/proto.c | 71 +--
src/backend/replication/logical/tablesync.c | 60 ++-
src/backend/replication/pgoutput/pgoutput.c | 170 +++++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 17 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_publication.h | 5 +
src/include/replication/logicalproto.h | 18 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 508 ++++++++++++--------
src/test/regress/sql/publication.sql | 45 +-
src/test/subscription/t/031_column_list.pl | 4 +-
19 files changed, 698 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index f6344b3b79..577bcb4b71 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns are allowed to be replicated during logical replication
+ according to the <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>include_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 057c46f3f5..e2895209a1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6544,7 +6544,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5deac9..cd20bd469c 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7e5e357fd9..e937f8aed7 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -225,6 +225,41 @@ filter_partitions(List *table_infos)
}
}
+/*
+ * Returns true if the relation has column list associated with the
+ * publication, false if the relation has no column list associated with the
+ * publication.
+ */
+bool
+is_column_list_publication(Publication *pub, Oid relid)
+{
+ HeapTuple cftuple = NULL;
+ bool isnull = true;
+
+ if (pub->alltables)
+ return false;
+
+ cftuple = SearchSysCache2(PUBLICATIONRELMAP,
+ ObjectIdGetDatum(relid),
+ ObjectIdGetDatum(pub->oid));
+ if (HeapTupleIsValid(cftuple))
+ {
+ /* Lookup the column list attribute. */
+ (void) SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
+ Anum_pg_publication_rel_prattrs,
+ &isnull);
+ if (!isnull)
+ {
+ ReleaseSysCache(cftuple);
+ return true;
+ }
+
+ ReleaseSysCache(cftuple);
+ }
+
+ return false;
+}
+
/*
* Returns true if any schema is associated with the publication, false if no
* schema is associated with the publication.
@@ -500,8 +535,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* pub_collist_validate
* Process and validate the 'columns' list and ensure the columns are all
* valid to use for a publication. Checks for and raises an ERROR for
- * any; unknown columns, system columns, duplicate columns or generated
- * columns.
+ * any unknown columns, system columns, or duplicate columns.
*
* Looks up each column's attnum and returns a 0-based Bitmapset of the
* corresponding attnums.
@@ -511,7 +545,6 @@ pub_collist_validate(Relation targetrel, List *columns)
{
Bitmapset *set = NULL;
ListCell *lc;
- TupleDesc tupdesc = RelationGetDescr(targetrel);
foreach(lc, columns)
{
@@ -530,12 +563,6 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
- errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
- colname));
-
if (bms_is_member(attnum, set))
ereport(ERROR,
errcode(ERRCODE_DUPLICATE_OBJECT),
@@ -1006,6 +1033,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1213,7 +1241,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped || (att->attgenerated && !pub->pubgencols))
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef374e..0129db18c6 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2741..a13225fd79 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -30,10 +30,11 @@
#define TRUNCATE_RESTART_SEQS (1<<1)
static void logicalrep_write_attrs(StringInfo out, Relation rel,
- Bitmapset *columns);
+ Bitmapset *columns, bool pubgencols);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
TupleTableSlot *slot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
static void logicalrep_read_attrs(StringInfo in, LogicalRepRelation *rel);
static void logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple);
@@ -52,6 +53,28 @@ column_in_column_list(int attnum, Bitmapset *columns)
return (columns == NULL || bms_is_member(attnum, columns));
}
+/*
+ * Check if the column should be published.
+ */
+static bool
+should_publish_column(Form_pg_attribute att, Bitmapset *columns,
+ bool pubgencols)
+{
+ if (att->attisdropped)
+ return false;
+
+ /*
+ * Skip publishing generated columns if the option is not specified and if
+ * they are not included in the column list.
+ */
+ if (att->attgenerated && !pubgencols && !columns)
+ return false;
+
+ if (!column_in_column_list(att->attnum, columns))
+ return false;
+
+ return true;
+}
/*
* Write BEGIN to the output stream.
@@ -412,7 +435,8 @@ logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn)
*/
void
logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns)
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_INSERT);
@@ -424,7 +448,7 @@ logicalrep_write_insert(StringInfo out, TransactionId xid, Relation rel,
pq_sendint32(out, RelationGetRelid(rel));
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -457,7 +481,7 @@ logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup)
void
logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, TupleTableSlot *newslot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
pq_sendbyte(out, LOGICAL_REP_MSG_UPDATE);
@@ -478,11 +502,11 @@ logicalrep_write_update(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, 'O'); /* old tuple follows */
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
pq_sendbyte(out, 'N'); /* new tuple follows */
- logicalrep_write_tuple(out, rel, newslot, binary, columns);
+ logicalrep_write_tuple(out, rel, newslot, binary, columns, pubgencols);
}
/*
@@ -532,7 +556,7 @@ logicalrep_read_update(StringInfo in, bool *has_oldtuple,
void
logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
TupleTableSlot *oldslot, bool binary,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
Assert(rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT ||
rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL ||
@@ -552,7 +576,7 @@ logicalrep_write_delete(StringInfo out, TransactionId xid, Relation rel,
else
pq_sendbyte(out, 'K'); /* old key follows */
- logicalrep_write_tuple(out, rel, oldslot, binary, columns);
+ logicalrep_write_tuple(out, rel, oldslot, binary, columns, pubgencols);
}
/*
@@ -668,7 +692,7 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
*/
void
logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
- Bitmapset *columns)
+ Bitmapset *columns, bool pubgencols)
{
char *relname;
@@ -690,7 +714,7 @@ logicalrep_write_rel(StringInfo out, TransactionId xid, Relation rel,
pq_sendbyte(out, rel->rd_rel->relreplident);
/* send the attribute info */
- logicalrep_write_attrs(out, rel, columns);
+ logicalrep_write_attrs(out, rel, columns, pubgencols);
}
/*
@@ -767,7 +791,7 @@ logicalrep_read_typ(StringInfo in, LogicalRepTyp *ltyp)
*/
static void
logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
- bool binary, Bitmapset *columns)
+ bool binary, Bitmapset *columns, bool pubgencols)
{
TupleDesc desc;
Datum *values;
@@ -781,10 +805,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -802,10 +823,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
if (isnull[i])
@@ -923,7 +941,8 @@ logicalrep_read_tuple(StringInfo in, LogicalRepTupleData *tuple)
* Write relation attribute metadata to the stream.
*/
static void
-logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
+logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns,
+ bool pubgencols)
{
TupleDesc desc;
int i;
@@ -938,10 +957,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
nliveatts++;
@@ -959,10 +975,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
- continue;
-
- if (!column_in_column_list(att->attnum, columns))
+ if (!should_publish_column(att, columns, pubgencols))
continue;
/* REPLICA IDENTITY FULL means all columns are sent as part of key. */
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761392..fa123fda6d 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -791,19 +791,20 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
- LogicalRepRelation *lrel, List **qual)
+fetch_remote_table_info(char *nspname, char *relname, LogicalRepRelation *lrel,
+ List **qual, bool *remotegencolpresent)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
int natt;
ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -851,7 +852,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
@@ -873,8 +874,7 @@ fetch_remote_table_info(char *nspname, char *relname,
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
- " (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)"
- " THEN NULL ELSE gpt.attrs END)"
+ " (gpt.attrs)"
" FROM pg_publication p,"
" LATERAL pg_get_publication_tables(p.pubname) gpt,"
" pg_class c"
@@ -948,20 +948,21 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped"
" AND a.attrelid = %u"
- " ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
- lrel->remoteid);
+ " ORDER BY a.attnum", lrel->remoteid, lrel->remoteid);
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -1005,6 +1006,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 180000)
+ *remotegencolpresent |= DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1037,7 +1041,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1127,12 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &lrel, &qual,
+ &gencol_copy_needed);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1141,16 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1184,19 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * generated columns are published, because copy of generated columns
+ * is not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1254,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024563..ed578ec014 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -84,9 +84,6 @@ static bool publications_valid;
static List *LoadPublications(List *pubnames);
static void publication_invalidation_cb(Datum arg, int cacheid,
uint32 hashvalue);
-static void send_relation_and_attrs(Relation relation, TransactionId xid,
- LogicalDecodingContext *ctx,
- Bitmapset *columns);
static void send_repl_origin(LogicalDecodingContext *ctx,
RepOriginId origin_id, XLogRecPtr origin_lsn,
bool send_origin);
@@ -170,6 +167,9 @@ typedef struct RelationSyncEntry
*/
Bitmapset *columns;
+ /* Include publishing generated columns */
+ bool pubgencols;
+
/*
* Private context to store additional data for this entry - state for the
* row filter expressions, column list, etc.
@@ -213,6 +213,9 @@ static void init_rel_sync_cache(MemoryContext cachectx);
static void cleanup_rel_sync_cache(TransactionId xid, bool is_commit);
static RelationSyncEntry *get_rel_sync_entry(PGOutputData *data,
Relation relation);
+static void send_relation_and_attrs(Relation relation, TransactionId xid,
+ LogicalDecodingContext *ctx,
+ RelationSyncEntry *relentry);
static void rel_sync_cache_relation_cb(Datum arg, Oid relid);
static void rel_sync_cache_publication_cb(Datum arg, int cacheid,
uint32 hashvalue);
@@ -731,11 +734,11 @@ maybe_send_schema(LogicalDecodingContext *ctx,
{
Relation ancestor = RelationIdGetRelation(relentry->publish_as_relid);
- send_relation_and_attrs(ancestor, xid, ctx, relentry->columns);
+ send_relation_and_attrs(ancestor, xid, ctx, relentry);
RelationClose(ancestor);
}
- send_relation_and_attrs(relation, xid, ctx, relentry->columns);
+ send_relation_and_attrs(relation, xid, ctx, relentry);
if (data->in_streaming)
set_schema_sent_in_streamed_txn(relentry, topxid);
@@ -749,9 +752,10 @@ maybe_send_schema(LogicalDecodingContext *ctx,
static void
send_relation_and_attrs(Relation relation, TransactionId xid,
LogicalDecodingContext *ctx,
- Bitmapset *columns)
+ RelationSyncEntry *relentry)
{
TupleDesc desc = RelationGetDescr(relation);
+ Bitmapset *columns = relentry->columns;
int i;
/*
@@ -766,12 +770,19 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
continue;
+ /*
+ * Skip publishing generated columns if the option is not specified and
+ * if they are not included in the column list.
+ */
+ if (att->attgenerated && !relentry->pubgencols && !columns)
+ continue;
+
/* Skip this attribute if it's not present in the column list */
if (columns != NULL && !bms_is_member(att->attnum, columns))
continue;
@@ -782,7 +793,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
}
OutputPluginPrepareWrite(ctx, false);
- logicalrep_write_rel(ctx->out, xid, relation, columns);
+ logicalrep_write_rel(ctx->out, xid, relation, columns, relentry->pubgencols);
OutputPluginWrite(ctx, false);
}
@@ -1008,6 +1019,67 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
}
+/*
+ * If the table contains a generated column, check for any conflicting
+ * values of publish_generated_columns in the publications.
+ */
+static void
+pgoutput_pubgencol_init(PGOutputData *data, List *publications,
+ RelationSyncEntry *entry)
+{
+ Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ TupleDesc desc = RelationGetDescr(relation);
+ bool gencolpresent = false;
+ ListCell *lc;
+ bool first = true;
+
+ /* Check if there is any generated column present */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+ if (att->attgenerated)
+ {
+ gencolpresent = true;
+ break;
+ }
+ }
+
+ /* There is no generated columns to be published */
+ if (!gencolpresent)
+ {
+ entry->pubgencols = false;
+ return;
+ }
+
+ /*
+ * There may be a conflicting value for publish_generated_columns in the
+ * publications.
+ */
+ foreach(lc, publications)
+ {
+ Publication *pub = lfirst(lc);
+
+ /*
+ * The column list takes precedence over pubgencols, so skip checking
+ * column list publications.
+ */
+ if (is_column_list_publication(pub, entry->publish_as_relid))
+ continue;
+
+ if (first)
+ {
+ entry->pubgencols = pub->pubgencols;
+ first = false;
+ }
+ else if (entry->pubgencols != pub->pubgencols)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot use different values of publish_generated_columns for table \"%s.%s\" in different publications",
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
+ }
+}
+
/*
* Initialize the column list.
*/
@@ -1017,7 +1089,31 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
{
ListCell *lc;
bool first = true;
+ Bitmapset *relcols = NULL;
Relation relation = RelationIdGetRelation(entry->publish_as_relid);
+ TupleDesc desc = RelationGetDescr(relation);
+ MemoryContext oldcxt = NULL;
+ bool collistpubexist = false;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ /*
+ * Prepare the columns that will be published for FOR ALL TABLES and
+ * FOR TABLES IN SCHEMA publication.
+ */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attisdropped || (att->attgenerated && !entry->pubgencols))
+ continue;
+
+ relcols = bms_add_member(relcols, att->attnum);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
/*
* Find if there are any column lists for this relation. If there are,
@@ -1032,14 +1128,15 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
* need to check all the given publication-table mappings and report an
* error if any publications have a different column list.
*
- * FOR ALL TABLES and FOR TABLES IN SCHEMA imply "don't use column list".
+ * FOR ALL TABLES and FOR TABLES IN SCHEMA will use the relcols which is
+ * prepared according to the pubgencols option.
*/
foreach(lc, publications)
{
Publication *pub = lfirst(lc);
HeapTuple cftuple = NULL;
Datum cfdatum = 0;
- Bitmapset *cols = NULL;
+ Bitmapset *cols = relcols;
/*
* If the publication is FOR ALL TABLES then it is treated the same as
@@ -1071,35 +1168,9 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
/* Build the column list bitmap in the per-entry context. */
if (!pub_no_list) /* when not null */
{
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
-
- pgoutput_ensure_entry_cxt(data, entry);
-
- cols = pub_collist_to_bitmapset(cols, cfdatum,
+ collistpubexist = true;
+ cols = pub_collist_to_bitmapset(NULL, cfdatum,
entry->entry_cxt);
-
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
-
- if (att->attisdropped || att->attgenerated)
- continue;
-
- nliveatts++;
- }
-
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
}
ReleaseSysCache(cftuple);
@@ -1115,10 +1186,17 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
ereport(ERROR,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot use different column lists for table \"%s.%s\" in different publications",
- get_namespace_name(RelationGetNamespace(relation)),
- RelationGetRelationName(relation)));
+ get_namespace_name(RelationGetNamespace(relation)),
+ RelationGetRelationName(relation)));
} /* loop all subscribed publications */
+ /*
+ * If no column list publications exist, columns will be selected later
+ * based on the generated columns option.
+ */
+ if (!collistpubexist)
+ entry->columns = NULL;
+
RelationClose(relation);
}
@@ -1531,15 +1609,18 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
{
case REORDER_BUFFER_CHANGE_INSERT:
logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
logicalrep_write_update(ctx->out, xid, targetrel, old_slot,
- new_slot, data->binary, relentry->columns);
+ new_slot, data->binary, relentry->columns,
+ relentry->pubgencols);
break;
case REORDER_BUFFER_CHANGE_DELETE:
logicalrep_write_delete(ctx->out, xid, targetrel, old_slot,
- data->binary, relentry->columns);
+ data->binary, relentry->columns,
+ relentry->pubgencols);
break;
default:
Assert(false);
@@ -2213,6 +2294,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
/* Initialize the row filter */
pgoutput_row_filter_init(data, rel_publications, entry);
+ /* Initialize publish generated columns value */
+ pgoutput_pubgencol_init(data, rel_publications, entry);
+
/* Initialize the column list */
pgoutput_column_list_init(data, rel_publications, entry);
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ce..1d79865058 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4291,23 +4292,29 @@ getPublications(Archive *fout)
query = createPQExpBuffer();
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubgencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4327,6 +4334,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4351,6 +4359,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4432,6 +4442,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed5ad..c1552ead45 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830491..91a4c63744 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e718..7d78fceed6 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6233,7 +6233,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6268,6 +6268,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6357,6 +6361,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6373,6 +6378,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6386,6 +6392,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6437,6 +6446,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6451,6 +6462,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6461,6 +6474,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 1be0056af7..fad2277991 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2261,7 +2261,7 @@ match_previous_words(int pattern_id,
"CURRENT_SCHEMA");
/* ALTER PUBLICATION <name> SET ( */
else if (Matches("ALTER", "PUBLICATION", MatchAny, MatchAnyN, "SET", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
@@ -3513,7 +3513,7 @@ match_previous_words(int pattern_id,
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (Matches("CREATE", "PUBLICATION", MatchAnyN, "WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a58b0..2839847ba0 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -150,6 +154,7 @@ extern Oid GetTopMostAncestorInPublication(Oid puboid, List *ancestors,
extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
+extern bool is_column_list_publication(Publication *pub, Oid relid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index c409638a2e..fa6d66bff8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -223,20 +223,21 @@ extern void logicalrep_write_origin(StringInfo out, const char *origin,
XLogRecPtr origin_lsn);
extern char *logicalrep_read_origin(StringInfo in, XLogRecPtr *origin_lsn);
extern void logicalrep_write_insert(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *newslot,
- bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *newslot,
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_insert(StringInfo in, LogicalRepTupleData *newtup);
extern void logicalrep_write_update(StringInfo out, TransactionId xid,
- Relation rel,
- TupleTableSlot *oldslot,
- TupleTableSlot *newslot, bool binary, Bitmapset *columns);
+ Relation rel, TupleTableSlot *oldslot,
+ TupleTableSlot *newslot, bool binary,
+ Bitmapset *columns, bool pubgencols);
extern LogicalRepRelId logicalrep_read_update(StringInfo in,
bool *has_oldtuple, LogicalRepTupleData *oldtup,
LogicalRepTupleData *newtup);
extern void logicalrep_write_delete(StringInfo out, TransactionId xid,
Relation rel, TupleTableSlot *oldslot,
- bool binary, Bitmapset *columns);
+ bool binary, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
@@ -247,7 +248,8 @@ extern List *logicalrep_read_truncate(StringInfo in,
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
bool transactional, const char *prefix, Size sz, const char *message);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
- Relation rel, Bitmapset *columns);
+ Relation rel, Bitmapset *columns,
+ bool pubgencols);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, TransactionId xid,
Oid typoid);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5e25..62e4820ce9 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245ed0c..72943ef59a 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,9 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +736,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +923,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1131,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1172,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1253,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1266,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1295,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1321,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1392,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1403,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1424,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1436,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1448,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1459,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1470,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1481,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1512,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1524,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1606,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1627,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1755,84 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | t
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a)
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
+Tables:
+ "public.gencols" (a, gen1)
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5986..1ee322fc4f 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated column "d" can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1109,7 +1112,47 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- ======================================================
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the 'publish_generated_columns' parameter enabled or disabled for
+-- different scenarios with/without generated columns in column lists.
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+
+-- Generated columns in column list, when 'publish_generated_columns'=false
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+\dRp+ pub1
+-- Generated columns in column list, when 'publish_generated_columns'=true
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+\dRp+ pub2
+
+-- Generated columns in column list, then set 'publication_generate_columns'=false
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+\dRp+ pub2
+
+-- Remove generated columns from column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a);
+\dRp+ pub2
+
+-- Add generated columns in column list, when 'publish_generated_columns'=false
+ALTER PUBLICATION pub2 SET TABLE gencols(a, gen1);
+\dRp+ pub2
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5020..2480aa4f14 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
2.34.1
[text/x-patch] v42-0003-Tap-tests-for-generated-columns.patch (11.1K, ../../CALDaNm2wFZRzSJLcNi_uMZcSUWuZ8+kktc0n3Nfw9Fdti9WbVA@mail.gmail.com/3-v42-0003-Tap-tests-for-generated-columns.patch)
download | inline diff:
From 99d3a30a63c3e8e02b6518830c96d5592c86fe90 Mon Sep 17 00:00:00 2001
From: Shubham Khanna <[email protected]>
Date: Thu, 10 Oct 2024 11:25:52 +1100
Subject: [PATCH v42 3/3] Tap tests for generated columns
Add tests for the combinations of generated column replication.
Also test effect of 'publish_generated_columns' option true/false.
Author: Shubham Khanna
Reviewed-by: Vignesh C
---
src/test/subscription/t/011_generated.pl | 275 +++++++++++++++++++++++
1 file changed, 275 insertions(+)
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index 8b2e5f4708..f02999812e 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -96,4 +96,279 @@ is( $result, qq(1|22|
8|176|18
9|198|19), 'generated columns replicated with trigger');
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# =============================================================================
+# The following test cases exercise logical replication for the combinations
+# where there is a generated column on one or both sides of pub/sub:
+# - generated -> normal
+#
+# Furthermore, the combinations are tested using:
+# a publication pub1, on the 'postgres' database, with option publish_generated_columns=false.
+# a publication pub2, on the 'postgres' database, with option publish_generated_columns=true.
+# a subscription sub1, on the 'postgres' database for publication pub1.
+# a subscription sub2, on the 'test_pgc_true' database for publication pub2.
+# =============================================================================
+
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE test_pgc_true");
+
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int GENERATED ALWAYS AS (a * 2) STORED);
+ INSERT INTO tab_gen_to_nogen (a) VALUES (1), (2), (3);
+ CREATE PUBLICATION regress_pub1_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = false);
+ CREATE PUBLICATION regress_pub2_gen_to_nogen FOR TABLE tab_gen_to_nogen WITH (publish_generated_columns = true);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub1_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub1_gen_to_nogen WITH (copy_data = true);
+));
+
+# Create table and subscription.
+$node_subscriber->safe_psql(
+ 'test_pgc_true', qq(
+ CREATE TABLE tab_gen_to_nogen (a int, b int);
+ CREATE SUBSCRIPTION regress_sub2_gen_to_nogen CONNECTION '$publisher_connstr' PUBLICATION regress_pub2_gen_to_nogen WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub1_gen_to_nogen', 'postgres');
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'regress_sub2_gen_to_nogen', 'test_pgc_true');
+
+# Initial sync test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|
+2|
+3|), 'tab_gen_to_nogen initial sync, when publish_generated_columns=false');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen");
+is( $result, qq(1|2
+2|4
+3|6),
+ 'tab_gen_to_nogen initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO tab_gen_to_nogen VALUES (4), (5)");
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('regress_sub1_gen_to_nogen');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|
+2|
+3|
+4|
+5|),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=false'
+);
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('regress_sub2_gen_to_nogen');
+$result = $node_subscriber->safe_psql('test_pgc_true',
+ "SELECT a, b FROM tab_gen_to_nogen ORDER BY a");
+is( $result, qq(1|2
+2|4
+3|6
+4|8
+5|10),
+ 'tab_gen_to_nogen incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION regress_sub1_gen_to_nogen");
+$node_subscriber->safe_psql('test_pgc_true',
+ "DROP SUBSCRIPTION regress_sub2_gen_to_nogen");
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ DROP PUBLICATION regress_pub1_gen_to_nogen;
+ DROP PUBLICATION regress_pub2_gen_to_nogen;
+));
+$node_subscriber->safe_psql('test_pgc_true', "DROP table tab_gen_to_nogen");
+$node_subscriber->safe_psql('postgres', "DROP DATABASE test_pgc_true");
+
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when publish_generated_columns=true
+# =============================================================================
+
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data including generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen, tab_gen_to_gen2(gen1) WITH (publish_generated_columns=false);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen2 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen2 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=false.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|),
+ 'tab_gen_to_gen initial sync, when publish_generated_columns=false');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen2 initial sync, when publish_generated_columns=false');
+
+# Insert data to verify incremental replication
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen2 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=false.
+# Verify that column 'b' is not replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen ORDER BY a");
+is( $result, qq(1|
+1|
+2|
+3|),
+ 'tab_gen_to_gen incremental replication, when publish_generated_columns=false'
+);
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen2 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen2 incremental replication, when publish_generated_columns=false'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
+
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE PUBLICATION pub1 FOR table tab_gen_to_gen3, tab_gen_to_gen4(gen1) WITH (publish_generated_columns=true);
+));
+
+# Insert values into tables.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 (a) VALUES (1), (1);
+ INSERT INTO tab_gen_to_gen4 (a) VALUES (1), (1);
+));
+
+# Create table and subscription with copy_data=true.
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE tab_gen_to_gen3 (a int, gen1 int);
+ CREATE TABLE tab_gen_to_gen4 (a int, gen1 int);
+ CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = true);
+));
+
+# Wait for initial sync.
+$node_subscriber->wait_for_subscription_sync;
+$node_publisher->wait_for_catchup('sub1');
+
+# Initial sync test when publish_generated_columns=true.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2),
+ 'tab_gen_to_gen3 initial sync, when publish_generated_columns=true');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2),
+ 'tab_gen_to_gen4 initial sync, when publish_generated_columns=true');
+
+# Insert data to verify incremental replication.
+# Verify that column 'b' is replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ INSERT INTO tab_gen_to_gen3 VALUES (2), (3);
+ INSERT INTO tab_gen_to_gen4 VALUES (2), (3);
+));
+
+# Incremental replication test when publish_generated_columns=true.
+# Verify that column 'b' is replicated.
+$node_publisher->wait_for_catchup('sub1');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen3 ORDER BY a");
+is( $result, qq(1|2
+1|2
+2|4
+3|6),
+ 'tab_gen_to_gen3 incremental replication, when publish_generated_columns=true'
+);
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM tab_gen_to_gen4 ORDER BY a");
+is( $result, qq(|2
+|2
+|4
+|6),
+ 'tab_gen_to_gen4 incremental replication, when publish_generated_columns=true'
+);
+
+# cleanup
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub1");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub1");
+
done_testing();
--
2.34.1
[text/x-patch] v42-0002-DOCS-Generated-Column-Replication.patch (13.2K, ../../CALDaNm2wFZRzSJLcNi_uMZcSUWuZ8+kktc0n3Nfw9Fdti9WbVA@mail.gmail.com/4-v42-0002-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From dd7a8ce846bdc2df6a130852bd3a0820d86acaef Mon Sep 17 00:00:00 2001
From: Vignesh C <[email protected]>
Date: Thu, 24 Oct 2024 19:41:09 +0530
Subject: [PATCH v42 2/3] DOCS - Generated Column Replication.
This patch updates docs to describe the new feature allowing replication of generated
columns. This includes addition of a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By: Vignesh C
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 3 +-
doc/src/sgml/logical-replication.sgml | 290 +++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 4 +
4 files changed, 297 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 577bcb4b71..a13f19bdbe 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -517,7 +517,8 @@ CREATE TABLE people (
Generated columns are allowed to be replicated during logical replication
according to the <command>CREATE PUBLICATION</command> option
<link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>include_generated_columns</literal></link>.
+ <literal>include_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0c27..7a8524e825 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1404,6 +1404,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
of columns in the list is not preserved.
</para>
+ <para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
<para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
@@ -1567,6 +1575,288 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+CREATE TABLE
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+INSERT 0 3
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+CREATE PUBLICATION
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+CREATE TABLE
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+CREATE SUBSCRIPTION
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub(# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub(# d int GENERATED ALWAYS AS (b + 1) STORED);
+CREATE TABLE
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int GENERATED ALWAYS AS (b * 100) STORED);
+CREATE TABLE
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub(# c int,
+test_sub(# d int);
+CREATE TABLE
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+CREATE PUBLICATION
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+CREATE SUBSCRIPTION
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+INSERT 0 1
+test_pub=# INSERT INTO t2 VALUES (1,2);
+INSERT 0 1
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index e2895209a1..71b6b2a535 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -7477,7 +7477,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</variablelist>
<para>
- Next, one of the following submessages appears for each column (except generated columns):
+ Next, one of the following submessages appears for each column:
<variablelist>
<varlistentry>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd469c..c13cd4db74 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
2.34.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-10 05:23 ` Peter Smith <[email protected]>
5 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-10 05:23 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Here are some comments for TAP test patch v37-0003.
I’m not in favour of the removal of such a large number of
'combination' and other 'misc' tests. In the commit message, please
delete me as a "co-author" of this patch.
======
1.
Any description or comment that still mentions "all combinations" is
no longer valid:
(e.g. in the comment message)
Add tests for all combinations of generated column replication.
(e.g. in the test file)
# The following test cases exercise logical replication for all combinations
# where there is a generated column on one or both sides of pub/sub:
and
# Furthermore, all combinations are tested using:
======
2.
+# --------------------------------------------------
+# Testcase: generated -> normal
+# Publisher table has generated column 'b'.
+# Subscriber table has normal column 'b'.
+# --------------------------------------------------
+
Now that COPY for generated columns is already implemented in patch
0001, shouldn't this test be using 'copy_data' enabled, so it can test
replication both for initial tablesync as well as normal replication?
That was the whole point of having the "# XXX copy_data=false for now.
This will be changed later." reminder comment in this file.
======
3.
Previously there were some misc tests to ensure that a generated
column which was then altered using DROP EXPRESSION would work as
expected. The test scenario was commented like:
+# =============================================================================
+# Misc test.
+#
+# A "normal -> generated" replication fails, reporting an error that the
+# subscriber side column is missing.
+#
+# In this test case we use DROP EXPRESSION to change the subscriber generated
+# column into a normal column, then verify replication works ok.
+# =============================================================================
Now in patch v37 this test no longer exists. Why?
======
4.
+# =============================================================================
+# The following test cases demonstrate behavior of generated column replication
+# when publish_generated_colums=false/true:
+#
+# Test: column list includes gencols, when publish_generated_columns=false
+# Test: column list does not include gencols, when
publish_generated_columns=false
+#
+# Test: column list includes gencols, when publish_generated_columns=true
+# Test: column list does not include gencols, when
publish_generated_columns=true
+# Test: no column list, when publish_generated_columns=true
+# =============================================================================
These tests are currently only testing the initial tablesync
replication. Since the COPY logic is different from the normal
replication logic, I think it would be better to test some normal
replication records as well, to make sure both parts work
consistently. This comment applies to all of the following test cases.
~~~
5.
+# Create table and publications.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE nogen_to_gen3 (a int, b int, gen1 int GENERATED ALWAYS
AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) STORED);
+ CREATE TABLE nogen_to_gen4 (c int, d int, gen1 int GENERATED ALWAYS
AS (c * 2) STORED, gen2 int GENERATED ALWAYS AS (c * 2) STORED);
+ INSERT INTO nogen_to_gen3 VALUES (1, 1);
+ INSERT INTO nogen_to_gen4 VALUES (1, 1);
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen3, nogen_to_gen4(gen1)
WITH (publish_generated_columns=true);
+));
+
5a.
The code should do only what the comments say it does. So, the INSERTS
should be done separately after the CREATE PUBLICATION, but before the
CREATE SUBSCRIPTION. A similar change should be made for all of these
test cases.
# Insert some initial data
INSERT INTO nogen_to_gen3 VALUES (1, 1);
INSERT INTO nogen_to_gen4 VALUES (1, 1);
~
5b.
The tables are badly named. Why are they 'nogen_to_gen', when the
publisher side has generated cols and the subscriber side does not?
This problem seems repeated in multiple subsequent test cases.
~
6.
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen ORDER BY a");
+is($result, qq(1|1||2),
+ 'gen_to_nogen initial sync, when publish_generated_columns=false');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT * FROM gen_to_nogen2 ORDER BY c");
+is($result, qq(1|1||),
+ 'gen_to_nogen2 initial sync, when publish_generated_columns=false');
IMO all the "result" queries like these ones ought to have to have a
comment which explains the reason for the expected results. This
review comment applies to multiple places. Please add comments to all
of them.
~~~
7.
+# --------------------------------------------------
+# Testcase: Publisher replicates the column list data excluding generated
+# columns even though publish_generated_columns option is false.
+# --------------------------------------------------
+
7a.
This is the 2nd test case, but AFAICT it would be far easier to test
this scenario just by making another table (with an appropriate column
list) for the 1st test case.
~
7b.
BTW, I don't understand this test at all. I thought according to the
comment that it intended to use a publication column list with only
normal columns in it. But that is not what the publication looks like
here:
+ CREATE PUBLICATION pub1 FOR table nogen_to_gen, nogen_to_gen2(gen1)
WITH (publish_generated_columns=false);
Indeed, the way it is currently written I didn't see what this test is
doing that is any different from the prior test (???)
~~~
8.
+# --------------------------------------------------
+# Testcase: Although publish_generated_columns is true, publisher publishes
+# only the data of the columns specified in column list, skipping other
+# generated/non-generated columns.
+# --------------------------------------------------
versus
+# --------------------------------------------------
+# Testcase: Publisher publishes only the data of the columns specified in
+# column list skipping other generated/non-generated columns.
+# --------------------------------------------------
Again, I did not understand how these test cases differ from each
other. Surely, those can be combined easily enough just by adding
another table with a different kind of column list.
~~~
9.
+# --------------------------------------------------
+# Testcase: Publisher replicates all columns if publish_generated_columns is
+# enabled and there is no column list
+# --------------------------------------------------
+
Here is yet another test case that AFAICT can just be combined with
other test cases that were using publish_generated_columns=true. It
seems all you need is one extra table with no column list. You don't
need all the extra create/drop pub/sub overheads to test this.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-10-04 04:12 ` Peter Smith <[email protected]>
2024-10-08 06:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
1 sibling, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-10-04 04:12 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi Shubham, I don't have any new comments for the patch v36-0002.
But, according to my records, there are multiple old comments not yet
addressed for this patch. I am giving reminders for those below so
they don't get accidentally overlooked. Please re-confirm and at the
next posted version please respond individually to each of these to
say if they are addressed or not.
======
1. General
From review v31 [1] comment #1. Patches 0001 and 0002 should be merged.
======
src/backend/replication/logical/tablesync.c
make_copy_attnamelist:
2.
From review v31 [1] comment #4. Make the detailed useful error message
common if possible.
~~~
fetch_remote_table_info:
3.
From review v31 [1] comment #5. I was not sure if this logic is
sophisticated enough to handle the case when the same table has
gencols but there are multiple subscribed publications and the
'publish_generated_columns' parameter differs. Is this scenario
tested?
~
4.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
From review v32 [2] comment #1. This needs some careful testing. I was
not sure if sufficient to just check the 'publish_generated_columns'
flag. Now that "column lists take precedence" it is quite possible for
all publications to say 'publish_generated_columns=false', but the
publication can still publish gencols *anyway* if they are specified
in a column list.
======
[1] review v31 18/9 -
https://www.postgresql.org/message-id/flat/CAHv8Rj%2BKOoh58Uf5k2MN-%3DA3VdV60kCVKCh5ftqYxgkdxFSkqg%4...
[2] review v32 24/9 -
https://www.postgresql.org/message-id/CAHut%2BPu7EcK_JTgWS7GzeStHk6Asb1dmEzCJU2TJf%2BW1Zy30LQ%40mail...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:12 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-08 06:10 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-08 06:10 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Fri, Oct 4, 2024 at 9:43 AM Peter Smith <[email protected]> wrote:
>
> Hi Shubham, I don't have any new comments for the patch v36-0002.
>
> But, according to my records, there are multiple old comments not yet
> addressed for this patch. I am giving reminders for those below so
> they don't get accidentally overlooked. Please re-confirm and at the
> next posted version please respond individually to each of these to
> say if they are addressed or not.
>
> ======
>
> 1. General
> From review v31 [1] comment #1. Patches 0001 and 0002 should be merged.
>
> ======
> src/backend/replication/logical/tablesync.c
>
> make_copy_attnamelist:
>
> 2.
> From review v31 [1] comment #4. Make the detailed useful error message
> common if possible.
>
> ~~~
This comment is still open. Will fix this in next versions of patches.
>
> fetch_remote_table_info:
>
> 3.
> From review v31 [1] comment #5. I was not sure if this logic is
> sophisticated enough to handle the case when the same table has
> gencols but there are multiple subscribed publications and the
> 'publish_generated_columns' parameter differs. Is this scenario
> tested?
>
> ~
>
> 4.
> + * Get column lists for each relation, and check if any of the
> + * publications have the 'publish_generated_columns' parameter enabled.
>
> From review v32 [2] comment #1. This needs some careful testing. I was
> not sure if sufficient to just check the 'publish_generated_columns'
> flag. Now that "column lists take precedence" it is quite possible for
> all publications to say 'publish_generated_columns=false', but the
> publication can still publish gencols *anyway* if they are specified
> in a column list.
>
> ======
> [1] review v31 18/9 -
> https://www.postgresql.org/message-id/flat/CAHv8Rj%2BKOoh58Uf5k2MN-%3DA3VdV60kCVKCh5ftqYxgkdxFSkqg%4...
> [2] review v32 24/9 -
> https://www.postgresql.org/message-id/CAHut%2BPu7EcK_JTgWS7GzeStHk6Asb1dmEzCJU2TJf%2BW1Zy30LQ%40mail...
>
I have fixed the comments and posted the v37 patches for them. Please
refer to the updated v37 Patches here in [1]. See [1] for
the changes added.
[1] https://www.postgresql.org/message-id/CAHv8Rj%2BRnw%2B_SfSyyrvWL49AfJzx4O8YVvdU9gB%2BSQdt3%3DqF%2BA%...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-30 07:26 ` Peter Smith <[email protected]>
2024-10-03 11:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-30 07:26 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi Shubham. Here are my review comment for patch v34-0002.
======
doc/src/sgml/ref/create_publication.sgml
1.
+ <para>
+ This parameter can only be set <literal>true</literal> if
<literal>copy_data</literal> is
+ set to <literal>false</literal>.
+ </para>
Huh? AFAIK the patch implements COPY for generated columns, so why are
you saying this limitation?
======
src/backend/replication/logical/tablesync.c
2. reminder
Previously (18/9) [1 #4] I wrote maybe that other copy_data=false
"missing" case error can be improved to share the same error message
that you have in make_copy_attnamelist. And you replied [2] it would
be addressed in the next patchset, but that was at least 2 versions
back and I don't see any change yet.
======
[1] 18/9 review
https://www.postgresql.org/message-id/CAHut%2BPusbhvPrL1uN1TKY%3DFd4zu3h63eDebZvsF%3Duy%2BLBKTwgA%40...
[2] https://www.postgresql.org/message-id/CAHv8RjJ5_dmyCH58xQ0StXMdPt9gstemMMWytR79%2BLfOMAHdLw%40mail.g...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 07:26 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-03 11:04 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-10-03 11:04 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Mon, Sep 30, 2024 at 12:56 PM Peter Smith <[email protected]> wrote:
>
> Hi Shubham. Here are my review comment for patch v34-0002.
>
> ======
> doc/src/sgml/ref/create_publication.sgml
>
> 1.
> + <para>
> + This parameter can only be set <literal>true</literal> if
> <literal>copy_data</literal> is
> + set to <literal>false</literal>.
> + </para>
>
> Huh? AFAIK the patch implements COPY for generated columns, so why are
> you saying this limitation?
>
> ======
I have fixed this in the v36-0002 patch.
> src/backend/replication/logical/tablesync.c
>
> 2. reminder
>
> Previously (18/9) [1 #4] I wrote maybe that other copy_data=false
> "missing" case error can be improved to share the same error message
> that you have in make_copy_attnamelist. And you replied [2] it would
> be addressed in the next patchset, but that was at least 2 versions
> back and I don't see any change yet.
>
This comment is still open. Will fix this and post in the next version
of patches.
Please refer to the updated v36-0002 Patch here in [1]. See [1] for
the changes added.
[1] https://www.postgresql.org/message-id/CAHv8Rj%2B1RDd7AnJNzOJXk--zcbTtU3nys%3DZgU3ktB4e3DWbJgg%40mail...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-30 08:33 ` Peter Smith <[email protected]>
2024-10-03 00:09 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-30 08:33 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi, here is an updated patch set v35:
v35-0001 (same as v34-0001)
v35-0002 (same as v34-0002)
v35-0003 (updated)
- fixed a linkend problem of v34-0003.
- added info in "Column Lists" about generated cols and linked that to
the new docs section.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
Attachments:
[application/octet-stream] v35-0002-Support-replication-of-generated-column-during-i.patch (18.2K, ../../CAHut+PvoQS9HjcGFZrTHrUQZ8vzyfAcSgeTgQEoO_-f8CrhW4A@mail.gmail.com/2-v35-0002-Support-replication-of-generated-column-during-i.patch)
download | inline diff:
From 029c698f4933c351011f69169f07731ec326e0a5 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Mon, 30 Sep 2024 16:26:52 +1000
Subject: [PATCH v35] Support replication of generated column during initial
sync.
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
when (publish_generated_columns = false)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column:
Publisher generated column is not replicated. The subscriber column
will be filled with the subscriber-side default data.
* publisher generated column => subscriber generated column: Publisher
generated column is not replicated. The subscriber generated column
will be filed with the subscriber-side computed or default data.
---
doc/src/sgml/ref/create_publication.sgml | 4 +
src/backend/catalog/pg_subscription.c | 31 +++++
src/backend/commands/subscriptioncmds.c | 31 -----
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 197 +++++++++++++++++++++++-----
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
7 files changed, 203 insertions(+), 69 deletions(-)
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index cd20bd4..f9ecdee 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -231,6 +231,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
+ <para>
+ This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
+ set to <literal>false</literal>.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc915..fcfbf86 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc63..addf307 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -440,37 +440,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
-/*
* Check that the specified publications are present on the publisher.
*/
static void
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b..338b083 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761..22ebe40 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,24 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +930,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +986,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1034,28 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1067,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -1005,6 +1100,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1015,7 +1113,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1135,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1221,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1236,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1292,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1363,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec..158b444 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40..8cdb7af 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
--
1.8.3.1
[application/octet-stream] v35-0003-DOCS-Generated-Column-Replication.patch (12.4K, ../../CAHut+PvoQS9HjcGFZrTHrUQZ8vzyfAcSgeTgQEoO_-f8CrhW4A@mail.gmail.com/3-v35-0003-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From bf47be947033d295f813ae8a30ced7cd5a32dc3c Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Mon, 30 Sep 2024 18:28:33 +1000
Subject: [PATCH v35] DOCS - Generated Column Replication.
This patch adds a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By:
Discussion: https://www.postgresql.org/message-id/flat/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 277 +++++++++++++++++++++++++++++++
doc/src/sgml/ref/create_publication.sgml | 4 +
3 files changed, 283 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 7b9c349..192180d 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> parameter
- <link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>publish_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 98a7ad0..8e80a3e 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1405,6 +1405,14 @@ test_sub=# SELECT * FROM child ORDER BY a;
</para>
<para>
+ Generated columns can also be specified in a column list. This allows
+ generated columns to be published, regardless of the publication parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. See
+ <xref linkend="logical-replication-gencols"/> for details.
+ </para>
+
+ <para>
Specifying a column list when the publication also publishes
<link linkend="sql-createpublication-params-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>
is not supported.
@@ -1567,6 +1575,275 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index f9ecdee..2119262 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -235,6 +235,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
set to <literal>false</literal>.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
1.8.3.1
[application/octet-stream] v35-0001-Enable-support-for-publish_generated_columns-opt.patch (82.9K, ../../CAHut+PvoQS9HjcGFZrTHrUQZ8vzyfAcSgeTgQEoO_-f8CrhW4A@mail.gmail.com/4-v35-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 835aeb6c8136bf7576d200414bea564bf239de5d Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Mon, 30 Sep 2024 09:35:12 +1000
Subject: [PATCH v35] Enable support for 'publish_generated_columns' option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit supports the transmission of generated column information and data
alongside regular table changes. This behaviour is controlled by a new
PUBLICATION parameter ('publish_generated_columns').
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
When 'publish_generated_columns' is false, generated columns are not replicated.
But when generated columns are specified in PUBLICATION col-list, it is
replicated even the 'publish_generated_columns' is false.
There is a change in 'pg_publicataion' catalog so we need to
bump the catversion.
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 2 +-
doc/src/sgml/ref/create_publication.sgml | 12 +
src/backend/catalog/pg_publication.c | 17 +-
src/backend/commands/publicationcmds.c | 36 ++-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/pgoutput/pgoutput.c | 102 ++++--
src/bin/pg_dump/pg_dump.c | 21 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/pg_publication.h | 7 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 478 ++++++++++++++++------------
src/test/regress/sql/publication.sql | 38 ++-
src/test/subscription/t/031_column_list.pl | 4 +-
17 files changed, 491 insertions(+), 277 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 8ab0ddb..7b9c349 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 11b6456..56de72c 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6543,7 +6543,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<para>
Next, the following message part appears for each column included in
- the publication (except generated columns):
+ the publication:
</para>
<variablelist>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5de..cd20bd4 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,18 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2..583da09 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -420,7 +420,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pub->pubgencols);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -507,7 +508,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
{
Bitmapset *set = NULL;
ListCell *lc;
@@ -530,10 +531,10 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
- if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated && !pubgencols)
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list when publish_generated_columns as false",
colname));
if (bms_is_member(attnum, set))
@@ -1006,6 +1007,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1216,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef3..8c09125 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
@@ -1182,7 +1209,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2..6b085e5 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024..24c56ed 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1009,6 +1009,36 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
/*
+ * Prepare new column list bitmap.
+ * This encompasses all table columns, excluding the generated ones.
+ */
+static Bitmapset *
+prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
+/*
* Initialize the column list.
*/
static void
@@ -1042,13 +1072,14 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * To handle cases where the publish_generated_columns option isn't
+ * specified for all tables in a publication, the pubgencolumns check
+ * needs to be performed. In such cases, we must create a column list
+ * that excludes generated columns.
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
{
- bool pub_no_list = true;
+ bool pub_rel_has_collist = true;
/*
* Check for the presence of a column list in this publication.
@@ -1066,44 +1097,47 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
/* Lookup the column list attribute. */
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
- &pub_no_list);
+ &pub_rel_has_collist);
+ }
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ /* Build the column list bitmap in the per-entry context. */
+ if (!pub_rel_has_collist || !pub->pubgencols)
+ {
+ int i;
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- pgoutput_ensure_entry_cxt(data, entry);
+ pgoutput_ensure_entry_cxt(data, entry);
- cols = pub_collist_to_bitmapset(cols, cfdatum,
- entry->entry_cxt);
+ if (!pub_rel_has_collist)
+ cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
+ else
+ cols = prepare_all_columns_bms(data, entry, desc);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ /* Get the number of live attributes. */
+ for (i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
+ if (att->attisdropped)
+ continue;
- nliveatts++;
- }
+ nliveatts++;
+ }
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
}
+ }
+ if (HeapTupleIsValid(cftuple))
ReleaseSysCache(cftuple);
- }
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 130b807..a0dad1e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4293,23 +4294,29 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
else
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubviagencols "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -4326,6 +4333,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4350,6 +4358,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4429,6 +4439,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed..c1552ea 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830..91a4c63 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6a36c91..16cbef3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a7ccde6..ea36b18 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3182,7 +3182,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a5..2a3816f 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -152,7 +156,8 @@ extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool pubgencols);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5..62e4820 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245e..a26b5a4 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,10 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+WARNING: specified generated column "d" in publication column list when publish_generated_columns as false
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +737,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +924,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1132,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1173,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1254,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1267,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1296,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1322,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1393,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1404,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1425,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1437,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1449,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1460,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1471,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1482,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1513,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1525,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1607,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1628,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1756,53 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- gencols in column list with 'publish_generated_columns'=true.
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+-- No gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
+WARNING: "wal_level" is insufficient to publish logical changes
+HINT: Set "wal_level" to "logical" before creating subscriptions.
+-- ALTER PUBLICATION to add gencols to column list.
+ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+WARNING: specified generated column "gen1" in publication column list when publish_generated_columns as false
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP PUBLICATION pub3;
+DROP TABLE gencols;
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5..d7b43dc 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1110,6 +1113,39 @@ DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled for different combinations.
+SET client_min_messages = 'WARNING';
+CREATE TABLE gencols (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED);
+-- gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub1 FOR table gencols(a, gen1) WITH (publish_generated_columns=false);
+-- gencols in column list with 'publish_generated_columns'=true.
+CREATE PUBLICATION pub2 FOR table gencols(a, gen1) WITH (publish_generated_columns=true);
+-- ALTER PUBLICATION setting 'publication_generate_columns'=false.
+ALTER PUBLICATION pub2 SET (publish_generated_columns = false);
+
+-- No gencols in column list with 'publish_generated_columns'=false.
+CREATE PUBLICATION pub3 WITH (publish_generated_columns=false);
+-- ALTER PUBLICATION to add gencols to column list.
+ALTER PUBLICATION pub3 ADD TABLE gencols(a, gen1);
+
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+DROP PUBLICATION pub3;
+DROP TABLE gencols;
+
+RESET client_min_messages;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5..2480aa4 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,9 +1202,9 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
--
1.8.3.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 08:33 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-03 00:09 ` Peter Smith <[email protected]>
2024-10-03 00:53 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
0 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-10-03 00:09 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi Shubham,
The different meanings of the terms "parameter" versus "option" were
discussed in a recent thread [1], and that has made me reconsider this
generated columns feature.
Despite being in the PUBLICATION section "WITH ( publication_parameter
[= value] [, ... ] )", I think that 'publish_generated_columns' is an
"option" (not a parameter).
We should update all those places that are currently calling it a parameter:
- commit messages
- docs
- comments
- etc.
======
[1] https://www.postgresql.org/message-id/CAHut%2BPuiRydyrYfMzR1OxOnVJf-_G8OBCLdyqu8jJ8si51d%2BEQ%40mail...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 08:33 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 00:09 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-10-03 00:53 ` Peter Smith <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Peter Smith @ 2024-10-03 00:53 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: vignesh C <[email protected]>; Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Thu, Oct 3, 2024 at 10:09 AM Peter Smith <[email protected]> wrote:
>
> Hi Shubham,
>
> The different meanings of the terms "parameter" versus "option" were
> discussed in a recent thread [1], and that has made me reconsider this
> generated columns feature.
>
> Despite being in the PUBLICATION section "WITH ( publication_parameter
> [= value] [, ... ] )", I think that 'publish_generated_columns' is an
> "option" (not a parameter).
>
> We should update all those places that are currently calling it a parameter:
> - commit messages
> - docs
> - comments
> - etc.
>
> ======
> [1] https://www.postgresql.org/message-id/CAHut%2BPuiRydyrYfMzR1OxOnVJf-_G8OBCLdyqu8jJ8si51d%2BEQ%40mail...
>
It seems there are differing opinions on that other thread about what
term to use. Probably, it is best to just leave the above suggestion
alone for now.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-23 23:46 ` Peter Smith <[email protected]>
2024-09-27 14:42 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
3 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-23 23:46 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi. Here are my review comments for v32-0001
You wrote: "I have addressed all the comments in the v32-0001 Patch.",
however, I found multiple old review comments not addressed. Please
give a reason if a comment is deliberately left out, otherwise, I will
assume they are omitted by accident and so keep repeating them.
There were also still some unanswered questions from previous reviews,
so I have reminded you about those again here.
======
Commit message
1.
This commit enables support for the 'publish_generated_columns' option
in logical replication, allowing the transmission of generated column
information and data alongside regular table changes. The option
'publish_generated_columns' is a PUBLICATION parameter.
~
That PUBLICATION info in the 2nd sentence would be easier to say in
the 1st sentence.
SUGGESTION:
This commit supports the transmission of generated column information
and data alongside regular table changes. This behaviour is controlled
by a new PUBLICATION parameter ('publish_generated_columns').
~~~
2.
When 'publish_generated_columns' is false, generated columns are not
replicated, even when present in a PUBLICATION col-list.
Hm. This contradicts the behaviour that Amit wanted, (e.g.
"column-list takes precedence"). So I am not sure if this patch is
already catering for the behaviour suggested by Amit or if that is yet
to come in v33. For now, I am assuming that 32* has not caught up with
the latest behaviour requirements, but that might be a wrong
assumption; perhaps it is only this commit message that is bogus.
~~~
3. General.
On the same subject, there is lots of code, like:
if (att->attgenerated && !pub->pubgencols)
continue;
I suspect that might not be quite what you want for the "column-list
takes precedence" behaviour, but I am not going to identify all those
during this review. It needs lots of combinations of column list tests
to verify it.
======
doc/src/sgml/ddl.sgml
4ab.
nit - Huh?? Not changed the linkend as told in a previous review [1-#3a]
nit - Huh?? Not changed to call this a "parameter" instead of an
"option" as told in a previous review [1-#3b]
======
doc/src/sgml/protocol.sgml
5.
- <para>
- Next, the following message part appears for each column included in
- the publication (except generated columns):
- </para>
-
nit -- Huh?? I don't think you can just remove this whole paragraph.
But, probably you can just remove the "except generated columns" part.
I posted this same comment [4 #11] 20 patch versions back.
======
doc/src/sgml/ref/create_publication.sgml
6abc.
nit - Huh?? Not changed the parameter ID as told in a previous review [1-#6]
nit - Huh?? Not removed paragraph "This option is only available..."
as told in a previous review. See [1-#7]
nit - Huh?? Not removed paragraph "This parameter can only be set" as
told in a previous review. See [1-#8]
======
src/backend/catalog/pg_publication.c
7.
if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list
for publication with publish_generated_columns as false",
colname));
I did not understand how this WARNING can know
"publish_generated_columns as false"? Should the code be checking the
function parameter 'pubgencols'?
The errmsg also seemed a bit verbose. How about:
"specified generated column \"%s\" in publication column list when
publish_generated_columns = false"
======
src/backend/replication/logical/proto.c
8.
logicalrep_write_tuple:
logicalrep_write_attrs:
Reminder. I think I have multiple questions about this code from
previous reviews that may be still unanswered. See [2 #4]. Maybe when
you implement Amit's "column list takes precedence" behaviour then
this code is fine as-is (because the replication message might include
gencols or not-gecols regardless of the 'publish_generated_columns'
value). But I don't think that is the current implementation, so
something did not quite seem right. I am not sure. If you say it is
fine then I will believe it, but the question [2 #4] remains
unanswered.
======
src/backend/replication/pgoutput/pgoutput.c
9.
send_relation_and_attrs:
Reminder: Here is another question that was answered from [2 #5]. I
did not really trust it for the current implementation, but for the
"column list takes precedence" behaviour probably it will be ok.
~~~
10.
+/*
+ * Prepare new column list bitmap. This includes all the columns of the table.
+ */
+static Bitmapset *
+prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
This function needs a better comment with more explanation about what
this is REALLY doing. e.g. it says "includes all columns of the
table", but tthe implementation is skipping generated cols, so clearly
it is not "all columns of the table".
~~~
11. pgoutput_column_list_init
TBH, I struggle to read the logic of this function. Rewriting some
parts, inverting some variables, and adding more commentary might help
a lot.
11a.
There are too many "negatives" (with ! operator and with the word "no"
in the variable).
e.g. code is written in a backward way like:
if (!pub_no_list)
cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
else
cols = prepare_all_columns_bms(data, entry, desc);
instead of what could have been said:
if (pub_rel_has_collist)
cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
else
cols = prepare_all_columns_bms(data, entry, desc);
~
11b.
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * If the publication is FOR ALL TABLES and include generated columns
+ * then it is treated the same as if there are no column lists (even
+ * if other publications have a list).
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
The code does not appear to match the comment ("If the publication is
FOR ALL TABLES and include generated columns"). If it did it should
look like "if (pub->alltables && pub->pubgencols)".
Also, should "and include generated column" be properly referring to
the new PUBLICATION parameter name?
Also, the comment is somewhat confusing. I saw in the thread Vignesh
wrote an explanation like "To handle cases where the
publish_generated_columns option isn't specified for all tables in a
publication, the pubgencolumns check needs to be performed. In such
cases, we must create a column list that excludes generated columns"
[3]. IMO that was clearer information so something similar should be
written in this code comment.
~
11c.
+ /* Build the column list bitmap in the per-entry context. */
+ if (!pub_no_list || !pub->pubgencols) /* when not null */
I don't know what "when not null" means here. Aren't those both
booleans? How can it be "null"?
======
src/bin/pg_dump/pg_dump.c
12. getPublications:
Huh?? The code has not changed to address an old review comment I had
posted to say there seem multiple code fragments missing that should
say "false AS pubgencols". Refer to [2 #7].
======
src/bin/pg_dump/t/002_pg_dump.pl
13.
'ALTER PUBLICATION pub5 ADD TABLE test_table WHERE (col1 > 0);' => {
+ create_order => 51,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_table WHERE (col1 > 0);',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_table WHERE
((col1 > 0));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => {
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ },
+ },
+
+ 'ALTER PUBLICATION pub5 ADD TABLE test_second_table WHERE (col2 = \'test\');'
+ => {
+ create_order => 52,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_second_table
WHERE (col2 = \'test\');',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_second_table
WHERE ((col2 = 'test'::text));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => { exclude_dump_test_schema => 1, },
+ },
+
It wasn't clear to me how these tests are related to the patch.
Shouldn't there instead be some ALTER tests for trying to modify the
'publish_generate_columns' parameter?
======
src/test/regress/expected/publication.out
src/test/regress/sql/publication.sql
14.
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+WARNING: specified generated column "d" in publication column list
for publication with publish_generated_columns as false
I think these tests for the WARNING scenario need to be a bit more
deliberate. This seems to have happened as a side-effect. For example,
I was expecting more testing like:
Comments about various combinations to say what you are doing and what
you are expecting:
- gencols in column list with publish_generated_columns=false, expecting WARNING
- gencols in column list with publish_generated_columns=true, NOT
expecting WARNING
- gencols in column list with publish_generated_columns=true, then
ALTER PUBLICATION setting publication_generate_columns=false,
expecting WARNING
- NO gencols in column list with publish_generated_columns=false, then
ALTER PUBLICATION to add gencols to column list, expecting WARNING
======
src/test/subscription/t/031_column_list.pl
15.
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int
GENERATED ALWAYS AS (a + 1) STORED);
ALTER TABLE test_mix_4 DROP COLUMN c;
- CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 (a, b);
- CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4;
+ CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 WITH
(publish_generated_columns = true);
+ CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4 WITH
(publish_generated_columns = false);
I felt the comment for this test ought to be saying something more
about what you are doing with the 'publish_generated_columns'
parameters and what behaviour it was expecting.
======
Please refer to the attachment which addresses some of the nit
comments mentioned above.
======
[1] my review of v31-0001:
https://www.postgresql.org/message-id/CAHut%2BPsv-neEP_ftvBUBahh%2BKCWw%2BqQMF9N3sGU3YHWPEzFH-Q%40ma...
[2] my review of v30-0001:
https://www.postgresql.org/message-id/CAHut%2BPuaitgE4tu3nfaR%3DPCQEKjB%3DmpDtZ1aWkbwb%3DJZE8YvqQ%40...
[3] https://www.postgresql.org/message-id/CALDaNm1c7xPBodHw6LKp9e8hvGVJHcKH%3DDHK0iXmZuXKPnxZ3Q%40mail.g...
[4] https://www.postgresql.org/message-id/CAHut%2BPv45gB4cV%2BSSs6730Kb8urQyqjdZ9PBVgmpwqCycr1Ybg%40mail...
Kind Regards,
Peter Smith.
Fujitsu Australia
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 2e7804e..cca54bc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -515,8 +515,8 @@ CREATE TABLE people (
<listitem>
<para>
Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> option
- <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
<literal>publish_generated_columns</literal></link>.
</para>
</listitem>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 12ffcfb..56de72c 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6541,6 +6541,11 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</varlistentry>
</variablelist>
+ <para>
+ Next, the following message part appears for each column included in
+ the publication:
+ </para>
+
<variablelist>
<varlistentry>
<term>Int8</term>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index e133dc3..cd20bd4 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -223,7 +223,7 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
- <varlistentry id="sql-createpublication-params-with-include-generated-columns">
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
<term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
<listitem>
<para>
@@ -231,14 +231,6 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
- <para>
- This option is only available for replicating generated column data from the publisher
- to a regular, non-generated column in the subscriber.
- </para>
- <para>
- This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
- set to <literal>false</literal>.
- </para>
</listitem>
</varlistentry>
Attachments:
[text/plain] PS_NITPICKS_v320001.txt (2.4K, ../../CAHut+Pt9ArtYc1Vtb1DqzEEFwjcoDHMyRCnUqkHQeFJuMCDkvQ@mail.gmail.com/2-PS_NITPICKS_v320001.txt)
download | inline diff:
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 2e7804e..cca54bc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -515,8 +515,8 @@ CREATE TABLE people (
<listitem>
<para>
Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> option
- <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <command>CREATE PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
<literal>publish_generated_columns</literal></link>.
</para>
</listitem>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 12ffcfb..56de72c 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6541,6 +6541,11 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</varlistentry>
</variablelist>
+ <para>
+ Next, the following message part appears for each column included in
+ the publication:
+ </para>
+
<variablelist>
<varlistentry>
<term>Int8</term>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index e133dc3..cd20bd4 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -223,7 +223,7 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
- <varlistentry id="sql-createpublication-params-with-include-generated-columns">
+ <varlistentry id="sql-createpublication-params-with-publish-generated-columns">
<term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
<listitem>
<para>
@@ -231,14 +231,6 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
associated with the publication should be replicated.
The default is <literal>false</literal>.
</para>
- <para>
- This option is only available for replicating generated column data from the publisher
- to a regular, non-generated column in the subscriber.
- </para>
- <para>
- This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
- set to <literal>false</literal>.
- </para>
</listitem>
</varlistentry>
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 23:46 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-09-27 14:42 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-09-27 14:42 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, Sep 24, 2024 at 5:16 AM Peter Smith <[email protected]> wrote:
>
> Hi. Here are my review comments for v32-0001
>
> You wrote: "I have addressed all the comments in the v32-0001 Patch.",
> however, I found multiple old review comments not addressed. Please
> give a reason if a comment is deliberately left out, otherwise, I will
> assume they are omitted by accident and so keep repeating them.
>
> There were also still some unanswered questions from previous reviews,
> so I have reminded you about those again here.
>
> ======
> Commit message
>
> 1.
> This commit enables support for the 'publish_generated_columns' option
> in logical replication, allowing the transmission of generated column
> information and data alongside regular table changes. The option
> 'publish_generated_columns' is a PUBLICATION parameter.
>
> ~
>
> That PUBLICATION info in the 2nd sentence would be easier to say in
> the 1st sentence.
> SUGGESTION:
> This commit supports the transmission of generated column information
> and data alongside regular table changes. This behaviour is controlled
> by a new PUBLICATION parameter ('publish_generated_columns').
>
> ~~~
>
> 2.
> When 'publish_generated_columns' is false, generated columns are not
> replicated, even when present in a PUBLICATION col-list.
>
> Hm. This contradicts the behaviour that Amit wanted, (e.g.
> "column-list takes precedence"). So I am not sure if this patch is
> already catering for the behaviour suggested by Amit or if that is yet
> to come in v33. For now, I am assuming that 32* has not caught up with
> the latest behaviour requirements, but that might be a wrong
> assumption; perhaps it is only this commit message that is bogus.
>
> ~~~
>
> 3. General.
>
> On the same subject, there is lots of code, like:
>
> if (att->attgenerated && !pub->pubgencols)
> continue;
>
> I suspect that might not be quite what you want for the "column-list
> takes precedence" behaviour, but I am not going to identify all those
> during this review. It needs lots of combinations of column list tests
> to verify it.
>
> ======
> doc/src/sgml/ddl.sgml
>
> 4ab.
> nit - Huh?? Not changed the linkend as told in a previous review [1-#3a]
> nit - Huh?? Not changed to call this a "parameter" instead of an
> "option" as told in a previous review [1-#3b]
>
> ======
> doc/src/sgml/protocol.sgml
>
> 5.
> - <para>
> - Next, the following message part appears for each column included in
> - the publication (except generated columns):
> - </para>
> -
>
> nit -- Huh?? I don't think you can just remove this whole paragraph.
> But, probably you can just remove the "except generated columns" part.
> I posted this same comment [4 #11] 20 patch versions back.
>
> ======
> doc/src/sgml/ref/create_publication.sgml
>
> 6abc.
> nit - Huh?? Not changed the parameter ID as told in a previous review [1-#6]
> nit - Huh?? Not removed paragraph "This option is only available..."
> as told in a previous review. See [1-#7]
> nit - Huh?? Not removed paragraph "This parameter can only be set" as
> told in a previous review. See [1-#8]
>
> ======
> src/backend/catalog/pg_publication.c
>
> 7.
> if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
> - ereport(ERROR,
> + ereport(WARNING,
> errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
> - errmsg("cannot use generated column \"%s\" in publication column list",
> + errmsg("specified generated column \"%s\" in publication column list
> for publication with publish_generated_columns as false",
> colname));
>
> I did not understand how this WARNING can know
> "publish_generated_columns as false"? Should the code be checking the
> function parameter 'pubgencols'?
>
> The errmsg also seemed a bit verbose. How about:
> "specified generated column \"%s\" in publication column list when
> publish_generated_columns = false"
>
> ======
> src/backend/replication/logical/proto.c
>
> 8.
> logicalrep_write_tuple:
> logicalrep_write_attrs:
>
> Reminder. I think I have multiple questions about this code from
> previous reviews that may be still unanswered. See [2 #4]. Maybe when
> you implement Amit's "column list takes precedence" behaviour then
> this code is fine as-is (because the replication message might include
> gencols or not-gecols regardless of the 'publish_generated_columns'
> value). But I don't think that is the current implementation, so
> something did not quite seem right. I am not sure. If you say it is
> fine then I will believe it, but the question [2 #4] remains
> unanswered.
>
> ======
> src/backend/replication/pgoutput/pgoutput.c
>
> 9.
> send_relation_and_attrs:
>
> Reminder: Here is another question that was answered from [2 #5]. I
> did not really trust it for the current implementation, but for the
> "column list takes precedence" behaviour probably it will be ok.
>
> ~~~
>
> 10.
> +/*
> + * Prepare new column list bitmap. This includes all the columns of the table.
> + */
> +static Bitmapset *
> +prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
> + TupleDesc desc)
> +{
>
> This function needs a better comment with more explanation about what
> this is REALLY doing. e.g. it says "includes all columns of the
> table", but tthe implementation is skipping generated cols, so clearly
> it is not "all columns of the table".
>
> ~~~
>
> 11. pgoutput_column_list_init
>
> TBH, I struggle to read the logic of this function. Rewriting some
> parts, inverting some variables, and adding more commentary might help
> a lot.
>
> 11a.
> There are too many "negatives" (with ! operator and with the word "no"
> in the variable).
>
> e.g. code is written in a backward way like:
> if (!pub_no_list)
> cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
> else
> cols = prepare_all_columns_bms(data, entry, desc);
>
> instead of what could have been said:
> if (pub_rel_has_collist)
> cols = pub_collist_to_bitmapset(cols, cfdatum, entry->entry_cxt);
> else
> cols = prepare_all_columns_bms(data, entry, desc);
>
> ~
>
> 11b.
> - * If the publication is FOR ALL TABLES then it is treated the same as
> - * if there are no column lists (even if other publications have a
> - * list).
> + * If the publication is FOR ALL TABLES and include generated columns
> + * then it is treated the same as if there are no column lists (even
> + * if other publications have a list).
> */
> - if (!pub->alltables)
> + if (!pub->alltables || !pub->pubgencols)
>
> The code does not appear to match the comment ("If the publication is
> FOR ALL TABLES and include generated columns"). If it did it should
> look like "if (pub->alltables && pub->pubgencols)".
>
> Also, should "and include generated column" be properly referring to
> the new PUBLICATION parameter name?
>
> Also, the comment is somewhat confusing. I saw in the thread Vignesh
> wrote an explanation like "To handle cases where the
> publish_generated_columns option isn't specified for all tables in a
> publication, the pubgencolumns check needs to be performed. In such
> cases, we must create a column list that excludes generated columns"
> [3]. IMO that was clearer information so something similar should be
> written in this code comment.
> ~
>
> 11c.
> + /* Build the column list bitmap in the per-entry context. */
> + if (!pub_no_list || !pub->pubgencols) /* when not null */
>
> I don't know what "when not null" means here. Aren't those both
> booleans? How can it be "null"?
>
> ======
> src/bin/pg_dump/pg_dump.c
>
> 12. getPublications:
>
> Huh?? The code has not changed to address an old review comment I had
> posted to say there seem multiple code fragments missing that should
> say "false AS pubgencols". Refer to [2 #7].
>
> ======
> src/bin/pg_dump/t/002_pg_dump.pl
>
> 13.
> 'ALTER PUBLICATION pub5 ADD TABLE test_table WHERE (col1 > 0);' => {
> + create_order => 51,
> + create_sql =>
> + 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_table WHERE (col1 > 0);',
> + regexp => qr/^
> + \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_table WHERE
> ((col1 > 0));\E
> + /xm,
> + like => { %full_runs, section_post_data => 1, },
> + unlike => {
> + exclude_dump_test_schema => 1,
> + exclude_test_table => 1,
> + },
> + },
> +
> + 'ALTER PUBLICATION pub5 ADD TABLE test_second_table WHERE (col2 = \'test\');'
> + => {
> + create_order => 52,
> + create_sql =>
> + 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_second_table
> WHERE (col2 = \'test\');',
> + regexp => qr/^
> + \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_second_table
> WHERE ((col2 = 'test'::text));\E
> + /xm,
> + like => { %full_runs, section_post_data => 1, },
> + unlike => { exclude_dump_test_schema => 1, },
> + },
> +
>
> It wasn't clear to me how these tests are related to the patch.
> Shouldn't there instead be some ALTER tests for trying to modify the
> 'publish_generate_columns' parameter?
>
> ======
> src/test/regress/expected/publication.out
> src/test/regress/sql/publication.sql
>
> 14.
> --- error: generated column "d" can't be in list
> +-- ok: generated columns can be in the list too
> ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
> -ERROR: cannot use generated column "d" in publication column list
> +WARNING: specified generated column "d" in publication column list
> for publication with publish_generated_columns as false
>
> I think these tests for the WARNING scenario need to be a bit more
> deliberate. This seems to have happened as a side-effect. For example,
> I was expecting more testing like:
>
> Comments about various combinations to say what you are doing and what
> you are expecting:
> - gencols in column list with publish_generated_columns=false, expecting WARNING
> - gencols in column list with publish_generated_columns=true, NOT
> expecting WARNING
> - gencols in column list with publish_generated_columns=true, then
> ALTER PUBLICATION setting publication_generate_columns=false,
> expecting WARNING
> - NO gencols in column list with publish_generated_columns=false, then
> ALTER PUBLICATION to add gencols to column list, expecting WARNING
>
> ======
> src/test/subscription/t/031_column_list.pl
>
> 15.
> -# TEST: Generated and dropped columns are not considered for the column list.
> +# TEST: Dropped columns are not considered for the column list.
> # So, the publication having a column list except for those columns and a
> -# publication without any column (aka all columns as part of the columns
> +# publication without any column list (aka all columns as part of the column
> # list) are considered to have the same column list.
> $node_publisher->safe_psql(
> 'postgres', qq(
> CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int
> GENERATED ALWAYS AS (a + 1) STORED);
> ALTER TABLE test_mix_4 DROP COLUMN c;
>
> - CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 (a, b);
> - CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4;
> + CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 WITH
> (publish_generated_columns = true);
> + CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4 WITH
> (publish_generated_columns = false);
>
> I felt the comment for this test ought to be saying something more
> about what you are doing with the 'publish_generated_columns'
> parameters and what behaviour it was expecting.
>
> ======
> Please refer to the attachment which addresses some of the nit
> comments mentioned above.
>
> ======
> [1] my review of v31-0001:
> https://www.postgresql.org/message-id/CAHut%2BPsv-neEP_ftvBUBahh%2BKCWw%2BqQMF9N3sGU3YHWPEzFH-Q%40ma...
> [2] my review of v30-0001:
> https://www.postgresql.org/message-id/CAHut%2BPuaitgE4tu3nfaR%3DPCQEKjB%3DmpDtZ1aWkbwb%3DJZE8YvqQ%40...
> [3] https://www.postgresql.org/message-id/CALDaNm1c7xPBodHw6LKp9e8hvGVJHcKH%3DDHK0iXmZuXKPnxZ3Q%40mail.g...
> [4] https://www.postgresql.org/message-id/CAHut%2BPv45gB4cV%2BSSs6730Kb8urQyqjdZ9PBVgmpwqCycr1Ybg%40mail...
>
I have addressed all the comments in the v34-0001 Patch. Please refer
to the updated v34-0001 Patch here in [1]. See [1] for the changes
added.
[1] https://www.postgresql.org/message-id/CAHv8RjJkUdYCdK_bL3yvEV%3DzKrA2dsnZYa1VMT2H5v0%2BqbaGbA%40mail...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-24 01:37 ` Peter Smith <[email protected]>
2024-09-27 14:46 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
3 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-24 01:37 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi. Here are my v32-0002 review comments:
======
src/backend/replication/logical/tablesync.c
1. fetch_remote_table_info
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
I am not 100% sure about this logic anymore. Maybe it is OK, but it
requires careful testing because with Amit's "column lists take
precedence" it is now possible for the publication to say
'publish_generated_columns=false', but the publication can still
publish gencols *anyway* if they were specified in a column list.
~~~
2.
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ StringInfo pub_names = makeStringInfo();
+
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
~
nit - The comment here seems misplaced.
~~~
3.
+ if (server_version >= 120000)
+ {
+ has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+ }
My previous review comment about this [1 #10] was:
Can the 'gencols_allowed' var be removed, and the condition just be
replaced with if (!has_pub_with_pubgencols)? It seems equivalent
unless I am mistaken.
nit - So the current v32 code is not what I was expecting. What I
meant was 'has_pub_with_pubgencols' can only be true if server_version
>= 180000, so I thought there was no reason to check it again. For
reference, I've changed it to like I meant in the nitpicks attachment.
Please see if that works the same.
======
[1] my review of v31-0002.
https://www.postgresql.org/message-id/CAHut%2BPusbhvPrL1uN1TKY%3DFd4zu3h63eDebZvsF%3Duy%2BLBKTwgA%40...
Kind Regards,
Peter Smith.
Fujitsu Australia
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 0e34d7c..9fed6b3 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -912,13 +912,12 @@ fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
- StringInfo pub_names = makeStringInfo();
-
get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
@@ -1047,13 +1046,8 @@ fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
" WHERE a.attnum > 0::pg_catalog.int2"
" AND NOT a.attisdropped", lrel->remoteid);
- if (server_version >= 120000)
- {
- has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
-
- if (!has_pub_with_pubgencols)
- appendStringInfo(&cmd, " AND a.attgenerated = ''");
- }
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
appendStringInfo(&cmd,
" AND a.attrelid = %u"
Attachments:
[text/plain] PS_NITPICKS_v320002.txt (1.3K, ../../CAHut+Pu7EcK_JTgWS7GzeStHk6Asb1dmEzCJU2TJf+W1Zy30LQ@mail.gmail.com/2-PS_NITPICKS_v320002.txt)
download | inline diff:
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 0e34d7c..9fed6b3 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -912,13 +912,12 @@ fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
+ StringInfo pub_names = makeStringInfo();
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
- StringInfo pub_names = makeStringInfo();
-
get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
@@ -1047,13 +1046,8 @@ fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
" WHERE a.attnum > 0::pg_catalog.int2"
" AND NOT a.attisdropped", lrel->remoteid);
- if (server_version >= 120000)
- {
- has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
-
- if (!has_pub_with_pubgencols)
- appendStringInfo(&cmd, " AND a.attgenerated = ''");
- }
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
appendStringInfo(&cmd,
" AND a.attrelid = %u"
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-24 01:37 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-09-27 14:46 ` Shubham Khanna <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Shubham Khanna @ 2024-09-27 14:46 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Tue, Sep 24, 2024 at 7:08 AM Peter Smith <[email protected]> wrote:
>
> Hi. Here are my v32-0002 review comments:
>
> ======
> src/backend/replication/logical/tablesync.c
>
> 1. fetch_remote_table_info
>
> /*
> - * Get column lists for each relation.
> + * Get column lists for each relation, and check if any of the
> + * publications have the 'publish_generated_columns' parameter enabled.
>
> I am not 100% sure about this logic anymore. Maybe it is OK, but it
> requires careful testing because with Amit's "column lists take
> precedence" it is now possible for the publication to say
> 'publish_generated_columns=false', but the publication can still
> publish gencols *anyway* if they were specified in a column list.
>
> ~~~
>
This comment is still open. Will fix this in the next version of patches.
> 2.
> /*
> * Fetch info about column lists for the relation (from all the
> * publications).
> */
> + StringInfo pub_names = makeStringInfo();
> +
> + get_publications_str(MySubscription->publications, pub_names, true);
> resetStringInfo(&cmd);
> appendStringInfo(&cmd,
> ~
>
> nit - The comment here seems misplaced.
>
> ~~~
>
> 3.
> + if (server_version >= 120000)
> + {
> + has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
> +
> + if (!has_pub_with_pubgencols)
> + appendStringInfo(&cmd, " AND a.attgenerated = ''");
> + }
>
> My previous review comment about this [1 #10] was:
> Can the 'gencols_allowed' var be removed, and the condition just be
> replaced with if (!has_pub_with_pubgencols)? It seems equivalent
> unless I am mistaken.
>
> nit - So the current v32 code is not what I was expecting. What I
> meant was 'has_pub_with_pubgencols' can only be true if server_version
> >= 180000, so I thought there was no reason to check it again. For
> reference, I've changed it to like I meant in the nitpicks attachment.
> Please see if that works the same.
>
> ======
> [1] my review of v31-0002.
> https://www.postgresql.org/message-id/CAHut%2BPusbhvPrL1uN1TKY%3DFd4zu3h63eDebZvsF%3Duy%2BLBKTwgA%40...
>
I have addressed all the comments in the v34-0002 Patch. Please refer
to the updated v34-0002 Patch here in [1]. See [1] for the changes
added.
[1] https://www.postgresql.org/message-id/CAHv8RjJkUdYCdK_bL3yvEV%3DzKrA2dsnZYa1VMT2H5v0%2BqbaGbA%40mail...
Thanks and Regards,
Shubham Khanna.
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
@ 2024-09-26 06:14 ` Peter Smith <[email protected]>
2024-09-27 00:03 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
3 siblings, 1 reply; 72+ messages in thread
From: Peter Smith @ 2024-09-26 06:14 UTC (permalink / raw)
To: Shubham Khanna <[email protected]>; +Cc: Amit Kapila <[email protected]>; Masahiko Sawada <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
Hi, I have written a new patch to document this feature.
The patch adds a new section to the "Logical Replication" chapter. It
applies atop the existing patches.
v33-0001 (same as v32-0001)
v33-0002 (same as v32-0002)
v33-0003 (new DOCS)
Review comments are welcome.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
Attachments:
[application/octet-stream] v33-0003-DOCS-Generated-Column-Replication.patch (11.7K, ../../CAHut+PspVAjTyZa8S89_kwR7gK9UAzAQgegykzff3sALLayzXA@mail.gmail.com/2-v33-0003-DOCS-Generated-Column-Replication.patch)
download | inline diff:
From dff8b657fed6d3e3d883e3aa8d7b34ffdb1fbf1f Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 26 Sep 2024 16:03:43 +1000
Subject: [PATCH v33] DOCS - Generated Column Replication.
This patch adds a new section "Generated Column Replication" to the
"Logical Replication" documentation chapter.
Author: Peter Smith
Reviewed By:
Discussion:
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 269 +++++++++++++++++++++++++++++++
doc/src/sgml/ref/create_publication.sgml | 4 +
3 files changed, 275 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 2e7804e..135bb08 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,10 +514,8 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns may be skipped during logical replication according to the
- <command>CREATE PUBLICATION</command> option
- <link linkend="sql-createpublication-params-with-include-generated-columns">
- <literal>publish_generated_columns</literal></link>.
+ Generated columns are not always published during logical replication. See
+ <xref linkend="logical-replication-gencols"/> for details.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index df62eb4..7923990 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1567,6 +1567,275 @@ test_sub=# SELECT * FROM t1 ORDER BY id;
</sect1>
+ <sect1 id="logical-replication-gencols">
+ <title>Generated Column Replication</title>
+
+ <para>
+ Typically, a table at the subscriber will be defined the same as the
+ publisher table, so if the publisher table has a <link linkend="ddl-generated-columns">
+ <literal>GENERATED column</literal></link> then the subscriber table will
+ have a matching generated column. In this case, it is always the subscriber
+ table generated column value that is used.
+ </para>
+ <para>
+ For example, note below that subscriber table generated column value comes from the
+ subscriber column's calculation.
+<programlisting>
+test_pub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a + 1) STORED);
+test_pub=# INSERT INTO tab_gen_to_gen VALUES (1),(2),(3);
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE tab_gen_to_gen;
+test_pub=# SELECT * FROM tab_gen_to_gen;
+ a | b
+---+---
+ 1 | 2
+ 2 | 3
+ 3 | 4
+(3 rows)
+
+test_sub=# CREATE TABLE tab_gen_to_gen (a int, b int GENERATED ALWAYS AS (a * 100) STORED);
+test_sub=# CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=test_pub' PUBLICATION pub1;
+test_sub=# SELECT * from tab_gen_to_gen;
+ a | b
+---+----
+ 1 | 100
+ 2 | 200
+ 3 | 300
+(3 rows)
+</programlisting>
+ </para>
+ <para>
+ In fact, prior to version 18.0, logical replication does not publish
+ <literal>GENERATED</literal> columns at all.
+ </para>
+ <para>
+ But, replicating a generated column to a regular column can sometimes be
+ desirable.
+ <tip>
+ <para>
+ This feature may be useful when replicating data to a
+ non-PostgreSQL database via plugin output, especially if the target database
+ does not support generated columns.
+ </para>
+ </tip>
+ </para>
+
+ <sect2 id="logical-replication-gencols-howto">
+ <title>How to Publish Generated Columns</title>
+
+ <para>
+ Generated columns are not published by default, but users can opt to
+ publish generated columns just like regular ones.
+ </para>
+ <para>
+ There are two ways to do this:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Enable the <command>PUBLICATION</command> parameter
+ <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <literal>publish_generated_columns</literal></link>. This instructs
+ PostgreSQL logical replication to publish current and future generated
+ columns of the publication's tables.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Specify a table <link linkend="logical-replication-col-lists">column list</link>
+ to explicity nominate which generated columns will be published.
+ </para>
+ <note>
+ <para>
+ When determining which table columns will be published, a column list
+ takes precedence, overriding the effect of the
+ <literal>publish_generated_columns</literal> parameter.
+ </para>
+ </note>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-behavior-summary">
+ <title>Behavior Summary</title>
+
+ <para>
+ The following table summarizes behavior when there are generated columns
+ involved in the logical replication. Results are shown for when
+ publishing generated columns is disabled (default), and for when it is
+ enabled.
+ </para>
+ <table id="logical-replication-gencols-table-summary">
+ <title>Replication Result Summary</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Publish generated columns?</entry><entry>Publisher table column</entry><entry>Subscriber table column</entry><entry>Result</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>Publisher table column is not replicated. Use the subscriber table generated column value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column is not replicated. Use the subscriber table regular column default value.</entry>
+ </row>
+ <row>
+ <entry>No</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>Publisher table column is not replicated. Nothing happens.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>GENERATED</entry><entry>ERROR. Not supported.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>regular</entry><entry>Publisher table column value is replicated to the subscriber table column.</entry>
+ </row>
+ <row>
+ <entry>Yes</entry><entry>GENERATED</entry><entry>--missing--</entry><entry>ERROR. The column is reported as missing from the subscriber table.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <warning>
+ <para>
+ There's currently no support for subscriptions comprising several
+ publications where the same table has been published with different column
+ lists. See <xref linkend="logical-replication-col-lists"/>.
+ </para>
+ <para>
+ This same situation can occur if one publication is publishing generated
+ columns, while another publication in the same subscription is not
+ publishing generated columns for the same table.
+ </para>
+ </warning>
+ </sect2>
+
+ <sect2 id="logical-replication-gencols-examples">
+ <title>Examples</title>
+
+ <para>
+ Setup the publisher and subscriber tables. Note that the subscriber
+ table columns have same names, but are not defined the same as the
+ publisher columns.
+<programlisting>
+test_pub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+
+test_pub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_pub-# c int GENERATED ALWAYS AS (a + 1) STORED,
+test_pub-# d int GENERATED ALWAYS AS (b + 1) STORED);
+</programlisting>
+<programlisting>
+test_sub=# CREATE TABLE t1 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int GENERATED ALWAYS AS (b * 100) STORED);
+
+test_sub=# CREATE TABLE t2 (a int PRIMARY KEY, b int,
+test_sub-# c int,
+test_sub-# d int);
+</programlisting>
+ </para>
+ <para>
+ Create the <literal>PUBLICATION</literal> and the <literal>SUBSCRIPTION</literal>.
+ Note that the publication specifies a column list for table <literal>t2</literal>.
+ The publication also sets parameter <literal>publish_generated_columns=false</literal>,
+ but that is just for demonstration because <literal>false</literal> is the
+ default anyway.
+<programlisting>
+test_pub=# CREATE PUBLICATION pub1 FOR TABLE t1, t2(a,c)
+test_pub-# WITH (publish_generated_columns=false);
+</programlisting>
+<programlisting>
+test_sub=# CREATE SUBSCRIPTION sub1
+test_sub-# CONNECTION 'dbname=test_pub'
+test_sub-# PUBLICATION pub1;
+</programlisting>
+ </para>
+ <para>
+ Insert some data to the publisher tables:
+<programlisting>
+test_pub=# INSERT INTO t1 VALUES (1,2);
+test_pub=# INSERT INTO t2 VALUES (1,2);
+
+test_pub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+
+test_pub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | 2 | 2 | 3
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t1</literal> were replicated:
+ <itemizedlist>
+ <listitem><para>
+ <literal>t1.a</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.b</literal> is a regular column. It gets replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.c</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.c</literal> default column value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t1.d</literal> is a generated column. It is not replicated because
+ <literal>publish_generated_columns=false</literal>. The subscriber
+ <literal>t2.d</literal> generated column value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t1;
+ a | b | c | d
+---+---+---+-----
+ 1 | 2 | | 200
+(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ Observe how columns for table <literal>t2</literal> were replicated.
+ <itemizedlist>
+ <listitem><para>
+ <literal>t2.a</literal> is a regular column. It was specified in the column
+ list, so is replicated normally.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.b</literal> is a regular column. It was not specified in column
+ list so is not replicated. The subscriber <literal>t2.b</literal> default
+ value is used.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.c</literal> is a generated column. It was specified in the
+ column list, so is replicated to the subscriber <literal>t2.c</literal>
+ regular column.
+ </para></listitem>
+ <listitem><para>
+ <literal>t2.d</literal> is a generated column. It was not specified in the
+ column list, so is not replicated. The subscriber <literal>t2.d</literal>
+ default value is used.
+ </para></listitem>
+ </itemizedlist>
+<programlisting>
+test_sub=# SELECT * FROM t2;
+ a | b | c | d
+---+---+---+---
+ 1 | | 2 |
+(1 row)
+</programlisting>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
<sect1 id="logical-replication-conflicts">
<title>Conflicts</title>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index 1973857..9b68557 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -235,6 +235,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
This option is only available for replicating generated column data from the publisher
to a regular, non-generated column in the subscriber.
</para>
+ <para>
+ See <xref linkend="logical-replication-gencols"/> for more details about
+ logical replication of generated columns.
+ </para>
</listitem>
</varlistentry>
--
1.8.3.1
[application/octet-stream] v33-0002-Support-replication-of-generated-column-during-i.patch (18.2K, ../../CAHut+PspVAjTyZa8S89_kwR7gK9UAzAQgegykzff3sALLayzXA@mail.gmail.com/3-v33-0002-Support-replication-of-generated-column-during-i.patch)
download | inline diff:
From 0144e78f6e1c467fdc8da3cc2fcdb07c97bce439 Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 26 Sep 2024 15:41:40 +1000
Subject: [PATCH v33] Support replication of generated column during initial
sync.
When 'copy_data' is true, during the initial sync, the data is replicated from
the publisher to the subscriber using the COPY command. The normal COPY
command does not copy generated columns, so when 'publish_generated_columns'
is true, we need to copy using the syntax:
'COPY (SELECT column_name FROM table_name) TO STDOUT'.
Here 'publish_generated_columns' is a PUBLICATION parameter and
'copy_data' is a SUBSCRIPTION parameter.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
Summary:
when (publish_generated_columns = true)
* publisher not-generated column => subscriber not-generated column:
This is just normal logical replication (not changed by this patch).
* publisher not-generated column => subscriber generated column: This
will give ERROR.
* publisher generated column => subscriber not-generated column: The
publisher generated column value is copied.
* publisher generated column => subscriber generated column: This
will give ERROR.
---
doc/src/sgml/ref/create_publication.sgml | 4 -
src/backend/catalog/pg_subscription.c | 31 +++++
src/backend/commands/subscriptioncmds.c | 31 -----
src/backend/replication/logical/relation.c | 2 +-
src/backend/replication/logical/tablesync.c | 203 +++++++++++++++++++++++-----
src/include/catalog/pg_subscription.h | 4 +
src/include/replication/logicalrelation.h | 3 +-
7 files changed, 205 insertions(+), 73 deletions(-)
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index e133dc3..1973857 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -235,10 +235,6 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
This option is only available for replicating generated column data from the publisher
to a regular, non-generated column in the subscriber.
</para>
- <para>
- This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
- set to <literal>false</literal>.
- </para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 9efc915..fcfbf86 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -551,3 +551,34 @@ GetSubscriptionRelations(Oid subid, bool not_ready)
return res;
}
+
+/*
+ * Add publication names from the list to a string.
+ */
+void
+get_publications_str(List *publications, StringInfo dest, bool quote_literal)
+{
+ ListCell *lc;
+ bool first = true;
+
+ Assert(publications != NIL);
+
+ foreach(lc, publications)
+ {
+ char *pubname = strVal(lfirst(lc));
+
+ if (first)
+ first = false;
+ else
+ appendStringInfoString(dest, ", ");
+
+ if (quote_literal)
+ appendStringInfoString(dest, quote_literal_cstr(pubname));
+ else
+ {
+ appendStringInfoChar(dest, '"');
+ appendStringInfoString(dest, pubname);
+ appendStringInfoChar(dest, '"');
+ }
+ }
+}
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 02ccc63..addf307 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -440,37 +440,6 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
}
/*
- * Add publication names from the list to a string.
- */
-static void
-get_publications_str(List *publications, StringInfo dest, bool quote_literal)
-{
- ListCell *lc;
- bool first = true;
-
- Assert(publications != NIL);
-
- foreach(lc, publications)
- {
- char *pubname = strVal(lfirst(lc));
-
- if (first)
- first = false;
- else
- appendStringInfoString(dest, ", ");
-
- if (quote_literal)
- appendStringInfoString(dest, quote_literal_cstr(pubname));
- else
- {
- appendStringInfoChar(dest, '"');
- appendStringInfoString(dest, pubname);
- appendStringInfoChar(dest, '"');
- }
- }
-}
-
-/*
* Check that the specified publications are present on the publisher.
*/
static void
diff --git a/src/backend/replication/logical/relation.c b/src/backend/replication/logical/relation.c
index f139e7b..338b083 100644
--- a/src/backend/replication/logical/relation.c
+++ b/src/backend/replication/logical/relation.c
@@ -205,7 +205,7 @@ logicalrep_relmap_update(LogicalRepRelation *remoterel)
*
* Returns -1 if not found.
*/
-static int
+int
logicalrep_rel_att_by_name(LogicalRepRelation *remoterel, const char *attname)
{
int i;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index e03e761..0e34d7c 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -118,6 +118,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
@@ -693,20 +694,72 @@ process_syncing_tables(XLogRecPtr current_lsn)
/*
* Create list of columns for COPY based on logical relation mapping.
+ * Exclude columns that are subscription table generated columns.
*/
static List *
-make_copy_attnamelist(LogicalRepRelMapEntry *rel)
+make_copy_attnamelist(LogicalRepRelMapEntry *rel, bool *remotegenlist)
{
List *attnamelist = NIL;
- int i;
+ bool *localgenlist;
+ TupleDesc desc;
- for (i = 0; i < rel->remoterel.natts; i++)
+ desc = RelationGetDescr(rel->localrel);
+
+ /*
+ * localgenlist stores if a generated column on remoterel has a matching
+ * name corresponding to a generated column on localrel.
+ */
+ localgenlist = palloc0(rel->remoterel.natts * sizeof(bool));
+
+ /*
+ * This loop checks for generated columns of the subscription table.
+ */
+ for (int i = 0; i < desc->natts; i++)
{
- attnamelist = lappend(attnamelist,
- makeString(rel->remoterel.attnames[i]));
+ int remote_attnum;
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+
+ if (!attr->attgenerated)
+ continue;
+
+ remote_attnum = logicalrep_rel_att_by_name(&rel->remoterel,
+ NameStr(attr->attname));
+
+ if (remote_attnum >= 0)
+ {
+ /*
+ * Check if the subscription table generated column has same name
+ * as a non-generated column in the corresponding publication
+ * table.
+ */
+ if (!remotegenlist[remote_attnum])
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("logical replication target relation \"%s.%s\" has a generated column \"%s\" "
+ "but corresponding column on source relation is not a generated column",
+ rel->remoterel.nspname, rel->remoterel.relname, NameStr(attr->attname))));
+
+ /*
+ * 'localgenlist' records that this is a generated column in the
+ * subscription table. Later, we use this information to skip
+ * adding this column to the column list for COPY.
+ */
+ localgenlist[remote_attnum] = true;
+ }
}
+ /*
+ * Construct column list for COPY, excluding columns that are subscription
+ * table generated columns.
+ */
+ for (int i = 0; i < rel->remoterel.natts; i++)
+ {
+ if (!localgenlist[i])
+ attnamelist = lappend(attnamelist,
+ makeString(rel->remoterel.attnames[i]));
+ }
+ pfree(localgenlist);
return attnamelist;
}
@@ -791,19 +844,21 @@ copy_read_data(void *outbuf, int minread, int maxread)
* qualifications to be used in the COPY command.
*/
static void
-fetch_remote_table_info(char *nspname, char *relname,
+fetch_remote_table_info(char *nspname, char *relname, bool **remotegenlist_res,
LogicalRepRelation *lrel, List **qual)
{
WalRcvExecResult *res;
StringInfoData cmd;
TupleTableSlot *slot;
Oid tableRow[] = {OIDOID, CHAROID, CHAROID};
- Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID};
+ Oid attrRow[] = {INT2OID, TEXTOID, OIDOID, BOOLOID, BOOLOID};
Oid qualRow[] = {TEXTOID};
bool isnull;
+ bool *remotegenlist;
+ bool has_pub_with_pubgencols = false;
int natt;
- ListCell *lc;
Bitmapset *included_cols = NULL;
+ int server_version = walrcv_server_version(LogRepWorkerWalRcvConn);
lrel->nspname = nspname;
lrel->relname = relname;
@@ -846,30 +901,25 @@ fetch_remote_table_info(char *nspname, char *relname,
/*
- * Get column lists for each relation.
+ * Get column lists for each relation, and check if any of the
+ * publications have the 'publish_generated_columns' parameter enabled.
*
* We need to do this before fetching info about column names and types,
* so that we can skip columns that should not be replicated.
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
WalRcvExecResult *pubres;
TupleTableSlot *tslot;
Oid attrsRow[] = {INT2VECTOROID};
- StringInfoData pub_names;
-
- initStringInfo(&pub_names);
- foreach(lc, MySubscription->publications)
- {
- if (foreach_current_index(lc) > 0)
- appendStringInfoString(&pub_names, ", ");
- appendStringInfoString(&pub_names, quote_literal_cstr(strVal(lfirst(lc))));
- }
/*
* Fetch info about column lists for the relation (from all the
* publications).
*/
+ StringInfo pub_names = makeStringInfo();
+
+ get_publications_str(MySubscription->publications, pub_names, true);
resetStringInfo(&cmd);
appendStringInfo(&cmd,
"SELECT DISTINCT"
@@ -881,7 +931,7 @@ fetch_remote_table_info(char *nspname, char *relname,
" WHERE gpt.relid = %u AND c.oid = gpt.relid"
" AND p.pubname IN ( %s )",
lrel->remoteid,
- pub_names.data);
+ pub_names->data);
pubres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
lengthof(attrsRow), attrsRow);
@@ -937,7 +987,44 @@ fetch_remote_table_info(char *nspname, char *relname,
walrcv_clear_result(pubres);
- pfree(pub_names.data);
+ /*
+ * Check if any of the publications have the
+ * 'publish_generated_columns' parameter enabled.
+ */
+ if (server_version >= 180000)
+ {
+ WalRcvExecResult *gencolres;
+ Oid gencolsRow[] = {BOOLOID};
+
+ resetStringInfo(&cmd);
+ appendStringInfo(&cmd,
+ "SELECT count(*) > 0 FROM pg_catalog.pg_publication "
+ "WHERE pubname IN ( %s ) AND pubgencols = 't'",
+ pub_names->data);
+
+ gencolres = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
+ lengthof(gencolsRow), gencolsRow);
+ if (gencolres->status != WALRCV_OK_TUPLES)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not fetch generated column publication information from publication list: %s",
+ pub_names->data));
+
+ tslot = MakeSingleTupleTableSlot(gencolres->tupledesc, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(gencolres->tuplestore, true, false, tslot))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("failed to fetch tuple for generated column publication information from publication list: %s",
+ pub_names->data));
+
+ has_pub_with_pubgencols = DatumGetBool(slot_getattr(tslot, 1, &isnull));
+ Assert(!isnull);
+
+ ExecClearTuple(tslot);
+ walrcv_clear_result(gencolres);
+ }
+
+ pfree(pub_names->data);
}
/*
@@ -948,20 +1035,33 @@ fetch_remote_table_info(char *nspname, char *relname,
"SELECT a.attnum,"
" a.attname,"
" a.atttypid,"
- " a.attnum = ANY(i.indkey)"
+ " a.attnum = ANY(i.indkey)");
+
+ if (server_version >= 180000)
+ appendStringInfo(&cmd, ", a.attgenerated != ''");
+
+ appendStringInfo(&cmd,
" FROM pg_catalog.pg_attribute a"
" LEFT JOIN pg_catalog.pg_index i"
" ON (i.indexrelid = pg_get_replica_identity_index(%u))"
" WHERE a.attnum > 0::pg_catalog.int2"
- " AND NOT a.attisdropped %s"
+ " AND NOT a.attisdropped", lrel->remoteid);
+
+ if (server_version >= 120000)
+ {
+ has_pub_with_pubgencols = server_version >= 180000 && has_pub_with_pubgencols;
+
+ if (!has_pub_with_pubgencols)
+ appendStringInfo(&cmd, " AND a.attgenerated = ''");
+ }
+
+ appendStringInfo(&cmd,
" AND a.attrelid = %u"
" ORDER BY a.attnum",
- lrel->remoteid,
- (walrcv_server_version(LogRepWorkerWalRcvConn) >= 120000 ?
- "AND a.attgenerated = ''" : ""),
lrel->remoteid);
+
res = walrcv_exec(LogRepWorkerWalRcvConn, cmd.data,
- lengthof(attrRow), attrRow);
+ server_version >= 180000 ? lengthof(attrRow) : lengthof(attrRow) - 1, attrRow);
if (res->status != WALRCV_OK_TUPLES)
ereport(ERROR,
@@ -973,6 +1073,7 @@ fetch_remote_table_info(char *nspname, char *relname,
lrel->attnames = palloc0(MaxTupleAttributeNumber * sizeof(char *));
lrel->atttyps = palloc0(MaxTupleAttributeNumber * sizeof(Oid));
lrel->attkeys = NULL;
+ remotegenlist = palloc0(MaxTupleAttributeNumber * sizeof(bool));
/*
* Store the columns as a list of names. Ignore those that are not
@@ -1005,6 +1106,9 @@ fetch_remote_table_info(char *nspname, char *relname,
if (DatumGetBool(slot_getattr(slot, 4, &isnull)))
lrel->attkeys = bms_add_member(lrel->attkeys, natt);
+ if (server_version >= 120000)
+ remotegenlist[natt] = DatumGetBool(slot_getattr(slot, 5, &isnull));
+
/* Should never happen. */
if (++natt >= MaxTupleAttributeNumber)
elog(ERROR, "too many columns in remote table \"%s.%s\"",
@@ -1015,7 +1119,7 @@ fetch_remote_table_info(char *nspname, char *relname,
ExecDropSingleTupleTableSlot(slot);
lrel->natts = natt;
-
+ *remotegenlist_res = remotegenlist;
walrcv_clear_result(res);
/*
@@ -1037,7 +1141,7 @@ fetch_remote_table_info(char *nspname, char *relname,
* 3) one of the subscribed publications is declared as TABLES IN SCHEMA
* that includes this relation
*/
- if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 150000)
+ if (server_version >= 150000)
{
StringInfoData pub_names;
@@ -1123,10 +1227,13 @@ copy_table(Relation rel)
List *attnamelist;
ParseState *pstate;
List *options = NIL;
+ bool *remotegenlist;
+ bool gencol_copy_needed = false;
/* Get the publisher relation info. */
fetch_remote_table_info(get_namespace_name(RelationGetNamespace(rel)),
- RelationGetRelationName(rel), &lrel, &qual);
+ RelationGetRelationName(rel), &remotegenlist,
+ &lrel, &qual);
/* Put the relation into relmap. */
logicalrep_relmap_update(&lrel);
@@ -1135,11 +1242,29 @@ copy_table(Relation rel)
relmapentry = logicalrep_rel_open(lrel.remoteid, NoLock);
Assert(rel == relmapentry->localrel);
+ attnamelist = make_copy_attnamelist(relmapentry, remotegenlist);
+
/* Start copy on the publisher. */
initStringInfo(&cmd);
- /* Regular table with no row filter */
- if (lrel.relkind == RELKIND_RELATION && qual == NIL)
+ /*
+ * Check if the remote table has any generated columns that should be
+ * copied.
+ */
+ for (int i = 0; i < relmapentry->remoterel.natts; i++)
+ {
+ if (remotegenlist[i])
+ {
+ gencol_copy_needed = true;
+ break;
+ }
+ }
+
+ /*
+ * Regular table with no row filter and copy of generated columns is not
+ * necessary.
+ */
+ if (lrel.relkind == RELKIND_RELATION && qual == NIL && !gencol_copy_needed)
{
appendStringInfo(&cmd, "COPY %s",
quote_qualified_identifier(lrel.nspname, lrel.relname));
@@ -1173,13 +1298,20 @@ copy_table(Relation rel)
* (SELECT ...), but we can't just do SELECT * because we need to not
* copy generated columns. For tables with any row filters, build a
* SELECT query with OR'ed row filters for COPY.
+ *
+ * We also need to use this same COPY (SELECT ...) syntax when
+ * 'publish_generated_columns' is specified as true and the remote
+ * table has generated columns, because copy of generated columns is
+ * not supported by the normal COPY.
*/
+ int i = 0;
+
appendStringInfoString(&cmd, "COPY (SELECT ");
- for (int i = 0; i < lrel.natts; i++)
+ foreach_node(String, att_name, attnamelist)
{
- appendStringInfoString(&cmd, quote_identifier(lrel.attnames[i]));
- if (i < lrel.natts - 1)
+ if (i++)
appendStringInfoString(&cmd, ", ");
+ appendStringInfoString(&cmd, quote_identifier(strVal(att_name)));
}
appendStringInfoString(&cmd, " FROM ");
@@ -1237,7 +1369,6 @@ copy_table(Relation rel)
(void) addRangeTableEntryForRelation(pstate, rel, AccessShareLock,
NULL, false, false);
- attnamelist = make_copy_attnamelist(relmapentry);
cstate = BeginCopyFrom(pstate, rel, NULL, NULL, false, copy_read_data, attnamelist, options);
/* Do the copy */
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 0aa14ec..158b444 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -20,6 +20,7 @@
#include "access/xlogdefs.h"
#include "catalog/genbki.h"
#include "catalog/pg_subscription_d.h"
+#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -180,4 +181,7 @@ extern void DisableSubscription(Oid subid);
extern int CountDBSubscriptions(Oid dbid);
+extern void get_publications_str(List *publications, StringInfo dest,
+ bool quote_literal);
+
#endif /* PG_SUBSCRIPTION_H */
diff --git a/src/include/replication/logicalrelation.h b/src/include/replication/logicalrelation.h
index e687b40..8cdb7af 100644
--- a/src/include/replication/logicalrelation.h
+++ b/src/include/replication/logicalrelation.h
@@ -41,7 +41,8 @@ typedef struct LogicalRepRelMapEntry
extern void logicalrep_relmap_update(LogicalRepRelation *remoterel);
extern void logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel);
-
+extern int logicalrep_rel_att_by_name(LogicalRepRelation *remoterel,
+ const char *attname);
extern LogicalRepRelMapEntry *logicalrep_rel_open(LogicalRepRelId remoteid,
LOCKMODE lockmode);
extern LogicalRepRelMapEntry *logicalrep_partition_open(LogicalRepRelMapEntry *root,
--
1.8.3.1
[application/octet-stream] v33-0001-Enable-support-for-publish_generated_columns-opt.patch (82.2K, ../../CAHut+PspVAjTyZa8S89_kwR7gK9UAzAQgegykzff3sALLayzXA@mail.gmail.com/4-v33-0001-Enable-support-for-publish_generated_columns-opt.patch)
download | inline diff:
From 3b24c5c7abe230a1944e95351d068b80e007230a Mon Sep 17 00:00:00 2001
From: Peter Smith <[email protected]>
Date: Thu, 26 Sep 2024 15:39:31 +1000
Subject: [PATCH v33] Enable support for 'publish_generated_columns' option.
Currently generated column values are not replicated because it is assumed
that the corresponding subscriber-side table will generate its own values
for those columns.
This commit enables support for the 'publish_generated_columns' option in
logical replication, allowing the transmission of generated column information
and data alongside regular table changes.
The option 'publish_generated_columns' is a PUBLICATION parameter.
When 'publish_generated_columns' is false, generated columns are not
replicated, even when present in a PUBLICATION col-list.
Example usage of subscription option:
CREATE PUBLICATION FOR TABLE tab_gencol WITH (publish_generated_columns
= true);
---
doc/src/sgml/ddl.sgml | 6 +-
doc/src/sgml/protocol.sgml | 5 -
doc/src/sgml/ref/create_publication.sgml | 20 ++
src/backend/catalog/pg_publication.c | 15 +-
src/backend/commands/publicationcmds.c | 36 ++-
src/backend/replication/logical/proto.c | 8 +-
src/backend/replication/pgoutput/pgoutput.c | 93 ++++--
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 36 +++
src/bin/psql/describe.c | 18 +-
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/pg_publication.h | 7 +-
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 452 +++++++++++++++-------------
src/test/regress/sql/publication.sql | 17 +-
src/test/subscription/t/031_column_list.pl | 36 +--
17 files changed, 475 insertions(+), 298 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index b671858..2e7804e 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -514,8 +514,10 @@ CREATE TABLE people (
</listitem>
<listitem>
<para>
- Generated columns are skipped for logical replication and cannot be
- specified in a <command>CREATE PUBLICATION</command> column list.
+ Generated columns may be skipped during logical replication according to the
+ <command>CREATE PUBLICATION</command> option
+ <link linkend="sql-createpublication-params-with-include-generated-columns">
+ <literal>publish_generated_columns</literal></link>.
</para>
</listitem>
</itemizedlist>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 11b6456..12ffcfb 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6541,11 +6541,6 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</varlistentry>
</variablelist>
- <para>
- Next, the following message part appears for each column included in
- the publication (except generated columns):
- </para>
-
<variablelist>
<varlistentry>
<term>Int8</term>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index fd9c5de..e133dc3 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -222,6 +222,26 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createpublication-params-with-include-generated-columns">
+ <term><literal>publish_generated_columns</literal> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether the generated columns present in the tables
+ associated with the publication should be replicated.
+ The default is <literal>false</literal>.
+ </para>
+ <para>
+ This option is only available for replicating generated column data from the publisher
+ to a regular, non-generated column in the subscriber.
+ </para>
+ <para>
+ This parameter can only be set <literal>true</literal> if <literal>copy_data</literal> is
+ set to <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7fe5fe2..cc12ef3 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -420,7 +420,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pub->pubgencols);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -507,7 +508,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool pubgencols)
{
Bitmapset *set = NULL;
ListCell *lc;
@@ -531,9 +532,9 @@ pub_collist_validate(Relation targetrel, List *columns)
colname));
if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated)
- ereport(ERROR,
+ ereport(WARNING,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use generated column \"%s\" in publication column list",
+ errmsg("specified generated column \"%s\" in publication column list for publication with publish_generated_columns as false",
colname));
if (bms_is_member(attnum, set))
@@ -1006,6 +1007,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubgencols = pubform->pubgencols;
ReleaseSysCache(tup);
@@ -1214,7 +1216,10 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
+ continue;
+
+ if (att->attgenerated && !pub->pubgencols)
continue;
attnums[nattnums++] = att->attnum;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index d6ffef3..8c09125 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -78,12 +78,15 @@ parse_publication_options(ParseState *pstate,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *publish_generated_columns_given,
+ bool *publish_generated_columns)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *publish_generated_columns_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -91,6 +94,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ *publish_generated_columns = false;
/* Parse options */
foreach(lc, options)
@@ -151,6 +155,13 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "publish_generated_columns") == 0)
+ {
+ if (*publish_generated_columns_given)
+ errorConflictingDefElem(defel, pstate);
+ *publish_generated_columns_given = true;
+ *publish_generated_columns = defGetBoolean(defel);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -737,6 +748,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
AclResult aclresult;
List *relations = NIL;
List *schemaidlist = NIL;
@@ -776,7 +789,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -793,6 +808,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubgencols - 1] =
+ BoolGetDatum(publish_generated_columns);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -878,6 +895,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
+ bool publish_generated_columns_given;
+ bool publish_generated_columns;
ObjectAddress obj;
Form_pg_publication pubform;
List *root_relids = NIL;
@@ -887,7 +906,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
stmt->options,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &publish_generated_columns_given,
+ &publish_generated_columns);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -997,6 +1018,12 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubviaroot - 1] = true;
}
+ if (publish_generated_columns_given)
+ {
+ values[Anum_pg_publication_pubgencols - 1] = BoolGetDatum(publish_generated_columns);
+ replaces[Anum_pg_publication_pubgencols - 1] = true;
+ }
+
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
replaces);
@@ -1182,7 +1209,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 980f6e2..6b085e5 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -781,7 +781,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -802,7 +802,7 @@ logicalrep_write_tuple(StringInfo out, Relation rel, TupleTableSlot *slot,
Form_pg_type typclass;
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -938,7 +938,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
@@ -959,7 +959,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel, Bitmapset *columns)
Form_pg_attribute att = TupleDescAttr(desc, i);
uint8 flags = 0;
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (!column_in_column_list(att->attnum, columns))
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00e7024..1f47ee7 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -766,7 +766,7 @@ send_relation_and_attrs(Relation relation, TransactionId xid,
{
Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
+ if (att->attisdropped)
continue;
if (att->atttypid < FirstGenbkiObjectId)
@@ -1009,6 +1009,35 @@ pgoutput_row_filter_init(PGOutputData *data, List *publications,
}
/*
+ * Prepare new column list bitmap. This includes all the columns of the table.
+ */
+static Bitmapset *
+prepare_all_columns_bms(PGOutputData *data, RelationSyncEntry *entry,
+ TupleDesc desc)
+{
+ Bitmapset *cols = NULL;
+ MemoryContext oldcxt = NULL;
+
+ pgoutput_ensure_entry_cxt(data, entry);
+ oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ /* Skip if the attribute is dropped or generated */
+ if (att->attisdropped || att->attgenerated)
+ continue;
+
+ cols = bms_add_member(cols, i + 1);
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+
+ return cols;
+}
+
+/*
* Initialize the column list.
*/
static void
@@ -1042,11 +1071,11 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
Bitmapset *cols = NULL;
/*
- * If the publication is FOR ALL TABLES then it is treated the same as
- * if there are no column lists (even if other publications have a
- * list).
+ * If the publication is FOR ALL TABLES and include generated columns
+ * then it is treated the same as if there are no column lists (even
+ * if other publications have a list).
*/
- if (!pub->alltables)
+ if (!pub->alltables || !pub->pubgencols)
{
bool pub_no_list = true;
@@ -1067,43 +1096,47 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
Anum_pg_publication_rel_prattrs,
&pub_no_list);
+ }
- /* Build the column list bitmap in the per-entry context. */
- if (!pub_no_list) /* when not null */
- {
- int i;
- int nliveatts = 0;
- TupleDesc desc = RelationGetDescr(relation);
+ /* Build the column list bitmap in the per-entry context. */
+ if (!pub_no_list || !pub->pubgencols) /* when not null */
+ {
+ int i;
+ int nliveatts = 0;
+ TupleDesc desc = RelationGetDescr(relation);
- pgoutput_ensure_entry_cxt(data, entry);
+ pgoutput_ensure_entry_cxt(data, entry);
+ if (!pub_no_list)
cols = pub_collist_to_bitmapset(cols, cfdatum,
entry->entry_cxt);
+ else
+ cols = prepare_all_columns_bms(data, entry, desc);
- /* Get the number of live attributes. */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
+ /* Get the number of live attributes. */
+ for (i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
- if (att->attisdropped || att->attgenerated)
- continue;
+ if (att->attisdropped)
+ continue;
- nliveatts++;
- }
+ nliveatts++;
+ }
- /*
- * If column list includes all the columns of the table,
- * set it to NULL.
- */
- if (bms_num_members(cols) == nliveatts)
- {
- bms_free(cols);
- cols = NULL;
- }
+ /*
+ * If column list includes all the columns of the table, set
+ * it to NULL.
+ */
+ if (bms_num_members(cols) == nliveatts)
+ {
+ bms_free(cols);
+ cols = NULL;
}
+ }
+ if (HeapTupleIsValid(cftuple))
ReleaseSysCache(cftuple);
- }
}
if (first)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 130b807..d1f0f36 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4282,6 +4282,7 @@ getPublications(Archive *fout)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubgencols;
int i,
ntups;
@@ -4293,7 +4294,13 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 180000)
+ appendPQExpBufferStr(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubgencols "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBufferStr(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
@@ -4326,6 +4333,7 @@ getPublications(Archive *fout)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubgencols = PQfnumber(res, "pubgencols");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -4350,6 +4358,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubgencols =
+ (strcmp(PQgetvalue(res, i, i_pubgencols), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4429,6 +4439,9 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
+ if (pubinfo->pubgencols)
+ appendPQExpBufferStr(query, ", publish_generated_columns = true");
+
appendPQExpBufferStr(query, ");\n");
if (pubinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9f907ed..c1552ea 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -626,6 +626,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubgencols;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ab6c830..2eb16f1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2986,6 +2986,16 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub5' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub5 WITH (publish_generated_columns = true);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub5 WITH (publish = 'insert, update, delete, truncate', publish_generated_columns = true);\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
@@ -3127,6 +3137,32 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'ALTER PUBLICATION pub5 ADD TABLE test_table WHERE (col1 > 0);' => {
+ create_order => 51,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_table WHERE (col1 > 0);',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_table WHERE ((col1 > 0));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => {
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ },
+ },
+
+ 'ALTER PUBLICATION pub5 ADD TABLE test_second_table WHERE (col2 = \'test\');'
+ => {
+ create_order => 52,
+ create_sql =>
+ 'ALTER PUBLICATION pub5 ADD TABLE dump_test.test_second_table WHERE (col2 = \'test\');',
+ regexp => qr/^
+ \QALTER PUBLICATION pub5 ADD TABLE ONLY dump_test.test_second_table WHERE ((col2 = 'test'::text));\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ unlike => { exclude_dump_test_schema => 1, },
+ },
+
'CREATE SCHEMA public' => {
regexp => qr/^CREATE SCHEMA public;/m,
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index faabecb..bfc978d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6237,7 +6237,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6272,7 +6272,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
-
+ if (pset.sversion >= 180000)
+ appendPQExpBuffer(&buf,
+ ",\n pubgencols AS \"%s\"",
+ gettext_noop("Generated columns"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6361,6 +6364,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubgencols;
PQExpBufferData title;
printTableContent cont;
@@ -6377,6 +6381,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubgencols = (pset.sversion >= 180000);
initPQExpBuffer(&buf);
@@ -6390,6 +6395,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubgencols)
+ appendPQExpBufferStr(&buf,
+ ", pubgencols");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6441,6 +6449,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubgencols)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6455,6 +6465,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubgencols)
+ printTableAddHeader(&cont, gettext_noop("Generated columns"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6465,6 +6477,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubgencols)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a7ccde6..ea36b18 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3182,7 +3182,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("WITH (");
/* Complete "CREATE PUBLICATION <name> [...] WITH" */
else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "("))
- COMPLETE_WITH("publish", "publish_via_partition_root");
+ COMPLETE_WITH("publish", "publish_generated_columns", "publish_via_partition_root");
/* CREATE RULE */
/* Complete "CREATE [ OR REPLACE ] RULE <sth>" with "AS ON" */
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index d9518a5..2a3816f 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if generated columns data should be published */
+ bool pubgencols;
} FormData_pg_publication;
/* ----------------
@@ -103,6 +106,7 @@ typedef struct Publication
char *name;
bool alltables;
bool pubviaroot;
+ bool pubgencols;
PublicationActions pubactions;
} Publication;
@@ -152,7 +156,8 @@ extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool pubgencols);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 3819bf5..62e4820 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6350,9 +6350,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+------+-------+------------+---------+---------+---------+-----------+----------+-------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 660245e..f060cef 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -29,21 +29,27 @@ CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publis
ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+ERROR: conflicting or redundant options
+LINE 1: ...pub_xxx WITH (publish_generated_columns = 'true', publish_ge...
+ ^
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
+ERROR: publish_generated_columns requires a Boolean value
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +93,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +105,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -123,10 +129,10 @@ CREATE PUBLICATION testpub_forschema FOR TABLES IN SCHEMA pub_test;
CREATE PUBLICATION testpub_for_tbl_schema FOR TABLES IN SCHEMA pub_test, TABLE pub_test.testpub_nopk;
RESET client_min_messages;
\dRp+ testpub_for_tbl_schema
- Publication testpub_for_tbl_schema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_for_tbl_schema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -144,10 +150,10 @@ LINE 1: ...CATION testpub_parsertst FOR TABLES IN SCHEMA foo, test.foo;
-- should be able to add a table of the same schema to the schema publication
ALTER PUBLICATION testpub_forschema ADD TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
Tables from schemas:
@@ -156,10 +162,10 @@ Tables from schemas:
-- should be able to drop the table
ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -170,10 +176,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -195,10 +201,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -210,19 +216,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -243,10 +249,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -261,10 +267,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -293,10 +299,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -309,10 +315,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -328,10 +334,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -339,10 +345,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -375,10 +381,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -388,10 +394,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -506,10 +512,10 @@ CREATE PUBLICATION testpub6 FOR TABLES IN SCHEMA testpub_rf_schema2;
ALTER PUBLICATION testpub6 SET TABLES IN SCHEMA testpub_rf_schema2, TABLE testpub_rf_schema2.testpub_rf_tbl6 WHERE (i < 99);
RESET client_min_messages;
\dRp+ testpub6
- Publication testpub6
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"testpub_rf_schema2.testpub_rf_tbl6" WHERE (i < 99)
Tables from schemas:
@@ -687,9 +693,10 @@ UPDATE testpub_tbl5 SET a = 1;
ERROR: cannot update table "testpub_tbl5"
DETAIL: Column list used by the publication does not cover the replica identity.
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
-ERROR: cannot use generated column "d" in publication column list
+WARNING: specified generated column "d" in publication column list for publication with publish_generated_columns as false
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ERROR: cannot use system column "ctid" in publication column list
@@ -730,10 +737,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -917,10 +924,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1125,10 +1132,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1166,10 +1173,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1247,10 +1254,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1260,20 +1267,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1289,19 +1296,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1315,44 +1322,44 @@ CREATE PUBLICATION testpub6_forschema FOR TABLES IN SCHEMA "CURRENT_SCHEMA", CUR
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1386,10 +1393,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1397,20 +1404,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1418,10 +1425,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1430,10 +1437,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1442,10 +1449,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1453,10 +1460,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1464,10 +1471,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1475,29 +1482,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1506,10 +1513,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1518,10 +1525,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1600,18 +1607,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1621,20 +1628,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR TABLES IN SCHEMA pub_test1, TA
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
@@ -1749,6 +1756,27 @@ DROP PUBLICATION pub;
DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+ Publication pub1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | t
+(1 row)
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+ Publication pub2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Generated columns
+--------------------------+------------+---------+---------+---------+-----------+----------+-------------------
+ regress_publication_user | t | t | t | t | t | f | f
+(1 row)
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index f68a5b5..51b6d46 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -24,6 +24,8 @@ ALTER PUBLICATION testpub_default SET (publish = update);
CREATE PUBLICATION testpub_xxx WITH (foo);
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
CREATE PUBLICATION testpub_xxx WITH (publish_via_partition_root = 'true', publish_via_partition_root = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'true', publish_generated_columns = '0');
+CREATE PUBLICATION testpub_xxx WITH (publish_generated_columns = 'foo');
\dRp
@@ -413,8 +415,9 @@ ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, x);
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (b, c);
UPDATE testpub_tbl5 SET a = 1;
ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
--- error: generated column "d" can't be in list
+-- ok: generated columns can be in the list too
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, d);
+ALTER PUBLICATION testpub_fortable DROP TABLE testpub_tbl5;
-- error: system attributes "ctid" not allowed in column list
ALTER PUBLICATION testpub_fortable ADD TABLE testpub_tbl5 (a, ctid);
ALTER PUBLICATION testpub_fortable SET TABLE testpub_tbl1 (id, ctid);
@@ -1110,6 +1113,18 @@ DROP TABLE sch1.tbl1;
DROP SCHEMA sch1 cascade;
DROP SCHEMA sch2 cascade;
+-- Test the publication 'publish_generated_columns' parameter enabled or disabled
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION pub1 FOR ALL TABLES WITH (publish_generated_columns=1);
+\dRp+ pub1
+
+CREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish_generated_columns=0);
+\dRp+ pub2
+
+RESET client_min_messages;
+DROP PUBLICATION pub1;
+DROP PUBLICATION pub2;
+
RESET SESSION AUTHORIZATION;
DROP ROLE regress_publication_user, regress_publication_user2;
DROP ROLE regress_publication_user_dummy;
diff --git a/src/test/subscription/t/031_column_list.pl b/src/test/subscription/t/031_column_list.pl
index 9a97fa5..68c7b29 100644
--- a/src/test/subscription/t/031_column_list.pl
+++ b/src/test/subscription/t/031_column_list.pl
@@ -1202,17 +1202,17 @@ $result = $node_publisher->safe_psql(
is( $result, qq(t
t), 'check the number of columns in the old tuple');
-# TEST: Generated and dropped columns are not considered for the column list.
+# TEST: Dropped columns are not considered for the column list.
# So, the publication having a column list except for those columns and a
-# publication without any column (aka all columns as part of the columns
+# publication without any column list (aka all columns as part of the column
# list) are considered to have the same column list.
$node_publisher->safe_psql(
'postgres', qq(
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int GENERATED ALWAYS AS (a + 1) STORED);
ALTER TABLE test_mix_4 DROP COLUMN c;
- CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 (a, b);
- CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4;
+ CREATE PUBLICATION pub_mix_7 FOR TABLE test_mix_4 WITH (publish_generated_columns = true);
+ CREATE PUBLICATION pub_mix_8 FOR TABLE test_mix_4 WITH (publish_generated_columns = false);
-- initial data
INSERT INTO test_mix_4 VALUES (1, 2);
@@ -1224,31 +1224,14 @@ $node_subscriber->safe_psql(
CREATE TABLE test_mix_4 (a int PRIMARY KEY, b int, c int, d int);
));
-$node_subscriber->safe_psql(
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql(
'postgres', qq(
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub_mix_7, pub_mix_8;
));
-$node_subscriber->wait_for_subscription_sync;
-
-is( $node_subscriber->safe_psql(
- 'postgres', "SELECT * FROM test_mix_4 ORDER BY a"),
- qq(1|2||),
- 'initial synchronization with multiple publications with the same column list'
-);
-
-$node_publisher->safe_psql(
- 'postgres', qq(
- INSERT INTO test_mix_4 VALUES (3, 4);
-));
-
-$node_publisher->wait_for_catchup('sub1');
-
-is( $node_subscriber->safe_psql(
- 'postgres', "SELECT * FROM test_mix_4 ORDER BY a"),
- qq(1|2||
-3|4||),
- 'replication with multiple publications with the same column list');
+ok( $stderr =~
+ qr/cannot use different column lists for table "public.test_mix_4" in different publications/,
+ 'different column lists detected');
# TEST: With a table included in multiple publications with different column
# lists, we should catch the error when creating the subscription.
@@ -1262,11 +1245,10 @@ $node_publisher->safe_psql(
$node_subscriber->safe_psql(
'postgres', qq(
- DROP SUBSCRIPTION sub1;
CREATE TABLE test_mix_1 (a int PRIMARY KEY, b int, c int);
));
-my ($cmdret, $stdout, $stderr) = $node_subscriber->psql(
+($cmdret, $stdout, $stderr) = $node_subscriber->psql(
'postgres', qq(
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub_mix_1, pub_mix_2;
));
--
1.8.3.1
^ permalink raw reply [nested|flat] 72+ messages in thread
* Re: Pgoutput not capturing the generated columns
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-26 06:14 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
@ 2024-09-27 00:03 ` Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 72+ messages in thread
From: Masahiko Sawada @ 2024-09-27 00:03 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Shubham Khanna <[email protected]>; Amit Kapila <[email protected]>; Rajendra Kumar Dangwal <[email protected]>; [email protected]; [email protected]
On Wed, Sep 25, 2024 at 11:15 PM Peter Smith <[email protected]> wrote:
>
> Hi, I have written a new patch to document this feature.
>
> The patch adds a new section to the "Logical Replication" chapter. It
> applies atop the existing patches.
>
> v33-0001 (same as v32-0001)
> v33-0002 (same as v32-0002)
> v33-0003 (new DOCS)
>
> Review comments are welcome.
Thank you for updating the patch!
I think that the patch doesn't have regression tests to check if
generated column data is replicated to the subscriber as expected. I
think we should include some tests for this feature (especially with
other features such as column list).
Also, when testing this feature, I got the following warning message
even if the publication has publish_generated_columns = true:
=# create publication pub for table test (a, c) with
(publish_generated_columns = true);
WARNING: specified generated column "c" in publication column list
for publication with publish_generated_columns as false
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 72+ messages in thread
end of thread, other threads:[~2024-10-28 13:10 UTC | newest]
Thread overview: 72+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-01-11 06:00 [PATCH 4/4] Documentation for slot-limit feature Kyotaro Horiguchi <[email protected]>
2018-01-11 06:00 [PATCH 5/6] Documentation for slot-limit feature Kyotaro Horiguchi <[email protected]>
2018-01-11 06:00 [PATCH 4/4] Documentation for slot-limit feature Kyotaro Horiguchi <[email protected]>
2018-01-11 06:00 [PATCH 5/6] Documentation for slot-limit feature Kyotaro Horiguchi <[email protected]>
2018-01-11 06:00 [PATCH 4/4] Documentation for slot-limit feature Kyotaro Horiguchi <[email protected]>
2019-02-21 09:35 [PATCH] Align timestamps in pg_regress output Christoph Berg <[email protected]>
2024-09-19 04:48 [PATCH v22 7/8] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]>
2024-09-20 11:45 Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-23 12:16 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-09-27 14:31 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 06:17 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:00 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-04 04:05 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:07 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 03:38 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:54 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-17 07:27 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:12 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-21 03:37 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-21 05:21 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-21 11:19 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 01:02 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-22 07:28 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-23 05:00 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-17 10:29 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-18 12:15 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 05:30 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 17:59 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 06:22 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-16 18:02 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-09 09:23 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-10 00:45 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-09 16:48 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-22 10:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-22 16:11 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
2024-10-23 06:21 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-23 06:55 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-23 08:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 06:45 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-24 11:14 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:20 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-25 06:37 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 10:24 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-25 14:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-25 14:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 02:13 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 05:33 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-28 06:56 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 08:50 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-28 11:43 ` RE: Pgoutput not capturing the generated columns Zhijie Hou (Fujitsu) <[email protected]>
2024-10-28 06:45 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-28 07:14 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-28 03:17 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
2024-10-28 13:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-10-28 06:54 ` RE: Pgoutput not capturing the generated columns Hayato Kuroda (Fujitsu) <[email protected]>
2024-10-28 11:45 ` RE: Pgoutput not capturing the generated columns Zhijie Hou (Fujitsu) <[email protected]>
2024-10-24 06:47 ` Re: Pgoutput not capturing the generated columns Amit Kapila <[email protected]>
2024-10-24 15:17 ` Re: Pgoutput not capturing the generated columns vignesh C <[email protected]>
2024-10-10 05:23 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-04 04:12 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-08 06:10 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 07:26 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 11:04 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-30 08:33 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 00:09 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-10-03 00:53 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-23 23:46 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-27 14:42 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-24 01:37 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-27 14:46 ` Re: Pgoutput not capturing the generated columns Shubham Khanna <[email protected]>
2024-09-26 06:14 ` Re: Pgoutput not capturing the generated columns Peter Smith <[email protected]>
2024-09-27 00:03 ` Re: Pgoutput not capturing the generated columns Masahiko Sawada <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox