Subject: Re: row filtering for logical replication
To: Stephen Frost <sfrost@snowman.net>,
 =?UTF-8?Q?Fabr=c3=adzio_de_Royes_Mello?= <fabriziomello@gmail.com>
Cc: petr.jelinek@2ndquadrant.com,
 Euler Taveira de Oliveira <euler@timbira.com.br>,
 Pgsql Hackers <pgsql-hackers@postgresql.org>, hironobu@interdb.jp
References: 
 <CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com>
 <CAHE3wgiHOdFHcEpm_w5f8vAXiugDmTciTiegNGVktezMbFkkrg@mail.gmail.com>
 <7106a0fc-8017-c0fe-a407-9466c9407ff8@2ndquadrant.com>
 <CAHE3wggWAtbT+Yt_MDy0uQwZcc6x820OwHp3_TBtAqtBeW4+Ug@mail.gmail.com>
 <92e5587d-28b8-5849-2374-5ca3863256f1@2ndquadrant.com>
 <CAFcNs+q88Z--uHbcii80r-pBWByEVNbc1nPyN8iHHKhCHCtzrA@mail.gmail.com>
 <466cdec7-92a0-41c0-b74a-ec0be94d4451@2ndquadrant.com>
 <CAFcNs+pA9ywYJbf940UFXW5B5i+An3=0yzxC8_bXubu+CsOQ=A@mail.gmail.com>
 <20181123190330.GI3415@tamriel.snowman.net>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <b2f86a5b-06fa-d1a9-1295-db673ebed550@2ndquadrant.com>
Date: Fri, 23 Nov 2018 21:00:46 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.0
MIME-Version: 1.0
In-Reply-To: <20181123190330.GI3415@tamriel.snowman.net>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 11/23/18 8:03 PM, Stephen Frost wrote:
> Greetings,
> 
> * Fabrízio de Royes Mello (fabriziomello@gmail.com) wrote:
>> On Fri, Nov 23, 2018 at 4:13 PM Petr Jelinek <petr.jelinek@2ndquadrant.com>
>> wrote:
>>>> If carefully documented I see no problem with it... we already have an
>>>> analogous problem with functional indexes.
>>>
>>> The difference is that with functional indexes you can recreate the
>>> missing object and everything is okay again. With logical replication
>>> recreating the object will not help.
>>>
>>
>> In this case with logical replication you should rsync the object. That is
>> the price of misunderstanding / bad use of the new feature.
>>
>> As usual, there are no free beer ;-)
> 
> There's also certainly no shortage of other ways to break logical
> replication, including ways that would also be hard to recover from
> today other than doing a full resync.
> 

Sure, but that seems more like an argument against creating additional
ones (and for preventing those that already exist). I'm not sure this
particular feature is where we should draw the line, though.

> What that seems to indicate, to me at least, is that it'd be awful
> nice to have a way to resync the data which doesn't necessairly
> involve transferring all of it over again.
> 
> Of course, it'd be nice if we could track those dependencies too,
> but that's yet another thing.

Yep, that seems like a good idea in general. Both here and for
functional indexes (although I suppose sure is a technical reason why it
wasn't implemented right away for them).

> 
> In short, I'm not sure that I agree with the idea that we shouldn't
> allow this and instead I'd rather we realize it and put the logical
> replication into some kind of an error state that requires a resync.
> 

That would still mean a need to resync the data to recover, so I'm not
sure it's really an improvement. And I suppose it'd require tracking the
dependencies, because how else would you mark the subscription as
requiring a resync? At which point we could decline the DROP without a
CASCADE, just like we do elsewhere, no?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services