public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Kapila <[email protected]>
To: Rahila Syed <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: Peter Smith <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: Column Filtering in Logical Replication
Date: Sat, 4 Sep 2021 10:12:53 +0530
Message-ID: <CAA4eK1LDLOy_DaZmwwscbknM51w5PEYeF65NfGHk-nZ1aTb54w@mail.gmail.com> (raw)
In-Reply-To: <CAH2L28t7GP7bLnoFa_uS_k9wfX7R2mqPz1-CtZBfJOOG+CM8aQ@mail.gmail.com>
References: <CAH2L28vddB_NFdRVpuyRBJEBWjz4BSyTB=_ektNRH8NJ1jf95g@mail.gmail.com>
<CAHut+PuP_hXzJQtLOvyaCxEPySPqhFheO2gZA3RcugMMfz3RSQ@mail.gmail.com>
<CAH2L28vZzcKQPV18EuwZG0wUr=E5g+E9WEbejtX-hEpjvJLLvg@mail.gmail.com>
<[email protected]>
<CAH2L28sY8031_udZBf6nTJwY01KPnSdPvWrPYWsE4c2=2RN8CQ@mail.gmail.com>
<CAH2L28stHzf6Yg3KxBEJ1JEr8C6xM4oSZfysdUObauZ00ArdWQ@mail.gmail.com>
<CAA4eK1+YTv8hELRS+10qBaPCvfV+wVHoWOqBmk3QRzrfg-Kzpw@mail.gmail.com>
<CAH2L28vtVkXj9cjY-SPFVfaHP0gKVLH77Rd2sz9LUJ=e2Pwchg@mail.gmail.com>
<CAH2L28t7GP7bLnoFa_uS_k9wfX7R2mqPz1-CtZBfJOOG+CM8aQ@mail.gmail.com>
On Thu, Sep 2, 2021 at 2:51 AM Rahila Syed <[email protected]> wrote:
>
>>>
>>> Do we want to consider that the columns specified in the filter must
>>> not have NOT NULL constraint? Because, otherwise, the subscriber will
>>> error out inserting such rows?
>>>
>> I think you mean columns *not* specified in the filter must not have NOT NULL constraint
>> on the subscriber, as this will break during insert, as it will try to insert NULL for columns
>> not sent by the publisher.
>> I will look into fixing this. Probably this won't be a problem in
>> case the column is auto generated or contains a default value.
>>
>
> I am not sure if this needs to be handled. Ideally, we need to prevent the subscriber tables from having a NOT NULL
> constraint if the publisher uses column filters to publish the values of the table. There is no way
> to do this at the time of creating a table on subscriber.
>
> As this involves querying the publisher for this information, it can be done at the time of initial table synchronization.
> i.e error out if any of the subscribed tables has NOT NULL constraint on non-filter columns.
> This will lead to the user dropping and recreating the subscription after removing the
> NOT NULL constraint from the table.
> I think the same can be achieved by doing nothing and letting the subscriber error out while inserting rows.
>
That makes sense and also it is quite possible that users don't have
such columns in the tables on subscribers. I guess we can add such a
recommendation in the docs instead of doing anything in the code.
Few comments:
============
1.
+
+ /*
+ * Cannot specify column filter when REPLICA IDENTITY IS FULL
+ * or if column filter does not contain REPLICA IDENITY columns
+ */
+ if (targetcols != NIL)
+ {
+ if (replidentfull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot add relation \"%s\" to publication",
+ RelationGetRelationName(targetrel)),
+ errdetail("Cannot have column filter with REPLICA IDENTITY FULL")));
Why do we want to have such a restriction for REPLICA IDENTITY FULL? I
think it is better to expand comments in that regards.
2.
@@ -839,7 +839,6 @@ NextCopyFrom(CopyFromState cstate, ExprContext *econtext,
ereport(ERROR,
(errcode(ERRCODE_BAD_COPY_FILE_FORMAT),
errmsg("extra data after last expected column")));
-
fieldno = 0;
@@ -944,7 +992,6 @@ logicalrep_write_attrs(StringInfo out, Relation rel)
flags |= LOGICALREP_IS_REPLICA_IDENTITY;
pq_sendbyte(out, flags);
-
/* attribute name */
pq_sendstring(out, NameStr(att->attname));
@@ -953,6 +1000,7 @@ logicalrep_write_attrs(StringInfo out, Relation rel)
/* attribute mode */
pq_sendint32(out, att->atttypmod);
+
}
Spurious line removals and addition.
--
With Regards,
Amit Kapila.
view thread (167+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Column Filtering in Logical Replication
In-Reply-To: <CAA4eK1LDLOy_DaZmwwscbknM51w5PEYeF65NfGHk-nZ1aTb54w@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox