From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
Cc: Rahila Syed <rahilasyed90@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: shiy.fnst@fujitsu.com
Subject: Re: Column Filtering in Logical Replication
Date: Thu, 10 Mar 2022 20:14:37 +0100
Message-ID: <5af7066b-15ac-1989-9afe-4e5dc1d60154@enterprisedb.com> (raw)
In-Reply-To: <CAA4eK1K5pkrPT9z5TByUPptExian5c18g6GnfNf9Cr97QdPbjw@mail.gmail.com>
References: <202201120041.p24wvsfcsope@alvherre.pgsql>
<ca91dc91-80ba-e954-213e-b4170a6160f5@enterprisedb.com>
<d37d6709-7e9c-f422-f925-fa66ce1fc0e8@enterprisedb.com>
<CAA4eK1KONbrck=hQTFSFgwio5rqu-82KZba2HLE_AcVa_RWYcQ@mail.gmail.com>
<cac80284-7196-6c77-adb9-ef0976a3e21f@enterprisedb.com>
<CAA4eK1JyoVty_BZOMJdx7VMkBTadvv8oOYrh9xTQ5BY+k-3+OA@mail.gmail.com>
<d26d24dd-2fab-3c48-0162-2b7f84a9c893@enterprisedb.com>
<CAA4eK1JJ5qgqbjfYQU9F4R1+OABxec-DQechQXus0fanDjaAVg@mail.gmail.com>
<CAA4eK1K5pkrPT9z5TByUPptExian5c18g6GnfNf9Cr97QdPbjw@mail.gmail.com>
On 3/10/22 04:09, Amit Kapila wrote:
> On Wed, Mar 9, 2022 at 3:33 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>>
>> On Mon, Mar 7, 2022 at 8:48 PM Tomas Vondra
>> <tomas.vondra@enterprisedb.com> wrote:
>>
>>> OK, I reworked this to do the same thing as the row filtering patch.
>>>
>>
>> Thanks, I'll check this.
>>
>
> Some assorted comments:
> =====================
> 1. We don't need to send a column list for the old tuple in case of an
> update (similar to delete). It is not required to apply a column
> filter for those cases because we ensure that RI must be part of the
> column list for updates and deletes.
I'm not sure which part of the code does this refer to?
> 2.
> + /*
> + * Check if all columns referenced in the column filter are part of
> + * the REPLICA IDENTITY index or not.
>
> I think this comment is reverse. The rule we follow here is that
> attributes that are part of RI must be there in a specified column
> list. This is used at two places in the patch.
Yeah, you're right. Will fix.
> 3. get_rel_sync_entry()
> {
> /* XXX is there a danger of memory leak here? beware */
> + oldctx = MemoryContextSwitchTo(CacheMemoryContext);
> + for (int i = 0; i < nelems; i++)
> ...
> }
>
> Similar to the row filter, I think we need to use
> entry->cache_expr_cxt to allocate this. There are other usages of
> CacheMemoryContext in this part of the code but I think those need to
> be also changed and we can do that as a separate patch. If we do the
> suggested change then we don't need to separately free columns.
I agree a shorter-lived context would be better than CacheMemoryContext,
but "expr" seems to indicate it's for the expression only, so maybe we
should rename that. But do we really want a memory context for every
single entry?
> 4. I think we don't need the DDL changes in AtExecDropColumn. Instead,
> we can change the dependency of columns to NORMAL during publication
> commands.
I'll think about that.
> 5. There is a reference to check_publication_columns but that function
> is removed from the patch.
Right, will fix.
> 6.
> /*
> * If we know everything is replicated and the row filter is invalid
> * for update and delete, there is no point to check for other
> * publications.
> */
> if (pubdesc->pubactions.pubinsert && pubdesc->pubactions.pubupdate &&
> pubdesc->pubactions.pubdelete && pubdesc->pubactions.pubtruncate &&
> !pubdesc->rf_valid_for_update && !pubdesc->rf_valid_for_delete)
> break;
>
> /*
> * If we know everything is replicated and the column filter is invalid
> * for update and delete, there is no point to check for other
> * publications.
> */
> if (pubdesc->pubactions.pubinsert && pubdesc->pubactions.pubupdate &&
> pubdesc->pubactions.pubdelete && pubdesc->pubactions.pubtruncate &&
> !pubdesc->cf_valid_for_update && !pubdesc->cf_valid_for_delete)
> break;
>
> Can we combine these two checks?
>
I was worried it'd get too complex / hard to understand, but I'll think
about maybe simplifying the conditions a bit.
> I feel this patch needs a more thorough review.
>
I won't object to more review, of course.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@enterprisedb.com, amit.kapila16@gmail.com, peter.eisentraut@enterprisedb.com, alvherre@alvh.no-ip.org, pryzby@telsasoft.com, houzj.fnst@fujitsu.com, rahilasyed90@gmail.com, smithpb2250@gmail.com, shiy.fnst@fujitsu.com
Subject: Re: Column Filtering in Logical Replication
In-Reply-To: <5af7066b-15ac-1989-9afe-4e5dc1d60154@enterprisedb.com>
References: <202201120041.p24wvsfcsope@alvherre.pgsql> <ca91dc91-80ba-e954-213e-b4170a6160f5@enterprisedb.com> <d37d6709-7e9c-f422-f925-fa66ce1fc0e8@enterprisedb.com> <CAA4eK1KONbrck=hQTFSFgwio5rqu-82KZba2HLE_AcVa_RWYcQ@mail.gmail.com> <cac80284-7196-6c77-adb9-ef0976a3e21f@enterprisedb.com> <CAA4eK1JyoVty_BZOMJdx7VMkBTadvv8oOYrh9xTQ5BY+k-3+OA@mail.gmail.com> <d26d24dd-2fab-3c48-0162-2b7f84a9c893@enterprisedb.com> <CAA4eK1JJ5qgqbjfYQU9F4R1+OABxec-DQechQXus0fanDjaAVg@mail.gmail.com> <CAA4eK1K5pkrPT9z5TByUPptExian5c18g6GnfNf9Cr97QdPbjw@mail.gmail.com> <5af7066b-15ac-1989-9afe-4e5dc1d60154@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox