Subject: Re: Column Filtering in Logical Replication
To: Alvaro Herrera <alvherre@alvh.no-ip.org>,
 Rahila Syed <rahilasyed90@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Peter Smith
 <smithpb2250@gmail.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <202109061751.3qz5xpugwx6w@alvherre.pgsql>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
Message-ID: <4804c901-3888-21cd-08ad-52c3e96858ed@enterprisedb.com>
Date: Mon, 6 Sep 2021 23:03:50 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <202109061751.3qz5xpugwx6w@alvherre.pgsql>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/4804c901-3888-21cd-08ad-52c3e96858ed%40enterprisedb.com>
Precedence: bulk


On 9/6/21 7:51 PM, Alvaro Herrera wrote:
> On 2021-Sep-06, Rahila Syed wrote:
> 
>>>> ... ugh.  Since CASCADE is already defined to be a
>>>> potentially-data-loss operation, then that may be acceptable
>>>> behavior.  For sure the default RESTRICT behavior shouldn't do it,
>>>> though.
>>>
>>> That makes sense to me.
>>
>> However, the default (RESTRICT) behaviour of DROP TABLE allows
>> removing the table from the publication. I have implemented the
>> removal of table from publication on drop column (RESTRICT)  on the
>> same lines.
> 
> But dropping the table is quite a different action from dropping a
> column, isn't it?  If you drop a table, it seems perfectly reasonable
> that it has to be removed from the publication -- essentially, when the
> user drops a table, she is saying "I don't care about this table
> anymore".  However, if you drop just one column, that doesn't
> necessarily mean that the user wants to stop publishing the whole table.
> Removing the table from the publication in ALTER TABLE DROP COLUMN seems
> like an overreaction.  (Except perhaps in the special case were the
> column being dropped is the only one that was being published.)
> 
> So let's discuss what should happen.  If you drop a column, and the
> column is filtered out, then it seems to me that the publication should
> continue to have the table, and it should continue to filter out the
> other columns that were being filtered out, regardless of CASCADE/RESTRICT.
> However, if the column is *included* in the publication, and you drop
> it, ISTM there are two cases:
> 
> 1. If it's DROP CASCADE, then the list of columns to replicate should
> continue to have all columns it previously had, so just remove the
> column that is being dropped.
> 
> 2. If it's DROP RESTRICT, then an error should be raised so that the
> user can make a concious decision to remove the column from the filter
> before dropping the column.
> 

FWIW I think this is a sensible behavior.

I don't quite see why dropping a column should remove the table from
publication (assuming there are some columns still replicated).

Of course, it may break the subscriber (e.g. when there was NOT NULL
constraint on that column), but DROP RESTRICT (which I assume is the
default mode) prevents that. And if DROP CASCADE is specified, I think
it's reasonable to require the user to fix the fallout.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company