Message-ID: <8b9860fc-bcd1-e579-a305-76f6f54cc98b@enterprisedb.com>
Date: Wed, 16 Feb 2022 01:39:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: Column Filtering in Logical Replication
Content-Language: en-US
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
 Justin Pryzby <pryzby@telsasoft.com>, Amit Kapila <amit.kapila16@gmail.com>,
 "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>,
 Rahila Syed <rahilasyed90@gmail.com>, Peter Smith <smithpb2250@gmail.com>,
 pgsql-hackers@postgresql.org
References: <202202160033.suwczmeq7mjf@alvherre.pgsql>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <202202160033.suwczmeq7mjf@alvherre.pgsql>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/8b9860fc-bcd1-e579-a305-76f6f54cc98b%40enterprisedb.com>
Precedence: bulk


On 2/16/22 01:33, Alvaro Herrera wrote:
> On 2022-Feb-16, Tomas Vondra wrote:
> 
>> Here's an updated version of the patch, rebased to current master. Parts
>> 0002 and 0003 include various improvements based on review by me and another
>> one by Peter Smith [1].
> 
> Thanks for doing this!
> 
>> 1) partitioning with pubviaroot=true
> 
> I agree that preventing the inconsistencies from happening is probably
> the best.
> 
>> 2) merging multiple column filters
>>
>> When the table has multiple column filters (in different publications), we
>> need to merge them. Which works, except that FOR ALL TABLES [IN SCHEMA]
>> needs to be handled as "has no column filter" (and replicates everything).
> 
> Agreed.
> 
>> 3) partitioning with pubivaroot=false
>>
>> When a partitioned table is added with (pubviaroot=false), it should not be
>> subject to column filter on the parent relation, which is the same behavior
>> used by the row filtering patch.
> 
> You mean each partition should define its own filter, or lack of filter?
> That sounds reasonable.
> 

If the partition is not published by the root, it shouldn't use the
filter defined on the root. I wonder what should happen to the filter
defined on the partition itself. I'd say

pubviaroot=false -> use filter defined on partition (if any)

pubviaroot=true -> use filter defined on root (if any)


I wonder what the row filter patch is doing - we should probably follow
the same logic, if only to keep the filtering stuff consistent.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company