MIME-Version: 1.0
References: <CAAWbhmiRRJDzbA4fv0vAViu_wfoeYqtT+VYOOu7845WO-yT35w@mail.gmail.com>
 <CAAWbhmiQWEGv73En15WRJ5PmhbwwTdBxMiPxunGghW0G-WVAYg@mail.gmail.com> <CAJ7c6TO+6hs1sMJ5-Qs7DC5HyhhN+hkLU0D8Ci_F-5ruBaHNog@mail.gmail.com>
In-Reply-To: <CAJ7c6TO+6hs1sMJ5-Qs7DC5HyhhN+hkLU0D8Ci_F-5ruBaHNog@mail.gmail.com>
From: Jacob Champion <jchampion@timescale.com>
Date: Tue, 10 Jan 2023 11:36:12 -0800
Message-ID: <CAAWbhmha62Vh9MtsJ3zxRKRQiVSoyxb6sKHzLCRWsTzqwM0bbg@mail.gmail.com>
Subject: Re: RFC: logical publication via inheritance root?
To: Aleksander Alekseev <aleksander@timescale.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://www.postgresql.org/message-id/CAAWbhmha62Vh9MtsJ3zxRKRQiVSoyxb6sKHzLCRWsTzqwM0bbg%40mail.gmail.com>
Precedence: bulk

On Mon, Jan 9, 2023 at 12:41 AM Aleksander Alekseev
<aleksander@timescale.com> wrote:
> I would like to point out that we shouldn't necessarily support
> multiple inheritance in all the possible cases, at least not in the
> first implementation. Supporting simple cases of inheritance would be
> already a valuable feature even if it will have certain limitations.

I agree. What I'm trying to avoid is the case where replication works
nicely for a table until someone performs an ALTER TABLE ... [NO]
INHERIT, and then Something Bad happens because we can't support the
new edge case. If every inheritance tree is automatically opted into
this new publication behavior, I think it'll be easier to hit that by
accident, making the whole thing feel brittle.

By contrast, if we have to opt specific tables into this feature by
marking them in the catalog, then not only will it be harder to hit by
accident (because we can document the requirements for the marker
function, and then it's up to the callers/extension authors/DBAs to
maintain those requirements), but we even have the chance to bail out
during an inheritance change if we see that the table is marked in
this way.

Two general pieces of progress to report:

1) I'm playing around with a marker in pg_inherits, where the inhseqno
is set to a sentinel value (0) for an inheritance relationship that
has been marked for logical publication. The intent is that the
pg_inherits helpers will prevent further inheritance relationships
when they see that marker, and reusing inhseqno means we can make use
of the existing index to do the lookups. An example:

    =# CREATE TABLE root (a int);
    =# CREATE TABLE root_p1 () INHERITS (root);
    =# SELECT pg_set_logical_root('root_p1', 'root');

and then any data written to root_p1 gets replicated via root instead,
if publish_via_partition_root = true. If root_p1 is set up with extra
columns, they'll be omitted from replication.

2) While this strategy works well for ongoing replication, it's not
enough to get the initial synchronization correct. The subscriber
still does a COPY of the root table directly, missing out on all the
logical descendant data. The publisher will have to tell the
subscriber about the relationship somehow, and older subscriber
versions won't understand how to use that (similar to how old
subscribers can't correctly handle row filters).

--Jacob