Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sX1jX-0013IF-4N for pgsql-hackers@arkaria.postgresql.org; Thu, 25 Jul 2024 16:54:27 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sX1jV-00H1WX-L3 for pgsql-hackers@arkaria.postgresql.org; Thu, 25 Jul 2024 16:54:25 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sWrFZ-00Bjtf-0D for pgsql-hackers@lists.postgresql.org; Thu, 25 Jul 2024 05:42:49 +0000 Received: from mail-qk1-x735.google.com ([2607:f8b0:4864:20::735]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sWrFW-001Jzd-2b for pgsql-hackers@lists.postgresql.org; Thu, 25 Jul 2024 05:42:47 +0000 Received: by mail-qk1-x735.google.com with SMTP id af79cd13be357-79f323f084eso28581485a.3 for ; Wed, 24 Jul 2024 22:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721886165; x=1722490965; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=gvYhFrtnIWmovkpusn9ZInSWSe0CJpdw2lqLQuiecMs=; b=djOnsyjmpmZXsKFQyvQhtUXEbwymDflm28GZHNwjZD7GxSvAbZX8CGsjUHRwrZfeOl JHu6J4O4Tg9uQsZoQj+JRnoxhAgb2Wel/mPzpOmGf1n/5sURqdWU+snVtzxvvDpPk3QU IM3KHq+QsqEvqiRDEdWx8MYdrQlOkBnCMIqHanjhgOMYazIDo8rCw1ehlLFFWmqzof1e m4Fdkti8y1sfq3nvwts2Vnmitf6gi6dgW+PK0AlTn8Vi6NnmzHa36eDVavyy3eQrKGgA pJKR82aEO49so76U9uAGxW8IGKeppe9ngfgtaIJ6wCU753WxwdGoLkiiQvGaK0PgIn7A ykVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721886165; x=1722490965; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gvYhFrtnIWmovkpusn9ZInSWSe0CJpdw2lqLQuiecMs=; b=oAlHVb/hICR7lvXeCWG6nZX7YFhDanBWI/ItkH2QWQWsUWE00kTUsJlzVD2yOvr8lo s0bqh+nda56+MS56Aqr/YYe5dvOXm4u2+ssgnscQpNM98c1sWizvI2jzjcpsV0bq53Fr LmU/9+y8N84Ku5gdfBmldgo2nVNGzZtdo1dcaGXpYzxbcmLq9pPcULytZKrQGpobLP5V /FksQ9MNfuN6Etl75QVhetdyJ3qmmPLmBmhYqRLA3dgaZXgp+9c0yPSWXzYquAbiG6TO hC1b1V2VIfkedUo/rLC6CZladwHLlxAli8411o0dwTFMLATVAHrzBlYPf9+8Ts2AgflT qdsg== X-Forwarded-Encrypted: i=1; AJvYcCXcLmOyNMl8jPRQtbz+3LCo4F7kBI7EOpD1CXZl3bxSHPc4v0Z7MNue0zJioWg6M+ZUaC875KUOeSyY2doPPee+okEXLcwIJhT0Qp+ERcKGSKqJ X-Gm-Message-State: AOJu0YyYO6648Whq1Ziclp5yUlz1vpCBSQ8xnemhtRuv07R4wsWbmIkB IZ5HJmQqq48qBv6ISy+8T/JsvPe9kY93VEuHwRGUA1w74wWWjNU4Bdbx9hduT1E31bDVHEFp5Pp NVNdPtCFpQu/HY4qffH8gOHHQmIA= X-Google-Smtp-Source: AGHT+IFANc1tO8MdMRb+eWxX0HH2XwJAK04iUSjZRX6388vbhD9iYIsNMHHoAKJSNAO10xgJ7Nsr6wi+QLQsJtfXllM= X-Received: by 2002:a05:6214:4015:b0:6b4:fcf7:2a31 with SMTP id 6a1803df08f44-6bb3ca5a2f9mr20991836d6.28.1721886165157; Wed, 24 Jul 2024 22:42:45 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Konstantin Berkaev Date: Thu, 25 Jul 2024 12:42:34 +0700 Message-ID: Subject: Re: Support logical replication of DDLs To: Amit Kapila Cc: "Andrey M. Borodin" , "Zhijie Hou (Fujitsu)" , Masahiko Sawada , shveta malik , Michael Paquier , "Wei Wang (Fujitsu)" , "Yu Shi (Fujitsu)" , vignesh C , Ajin Cherian , Runqi Tian , Peter Smith , Tom Lane , li jie , Dilip Kumar , Alvaro Herrera , Japin Li , rajesh singarapu , PostgreSQL Hackers , Zheng Li Content-Type: multipart/alternative; boundary="00000000000037d085061e0bdd3b" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000037d085061e0bdd3b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =D1=87=D1=82, 25 =D0=B8=D1=8E=D0=BB. 2024=E2=80=AF=D0=B3. =D0=B2 12:35, Ami= t Kapila : > On Thu, Mar 28, 2024 at 5:31=E2=80=AFPM Andrey M. Borodin > wrote: > > > > This thread is registered on CF [0] but is not active since 2023. Is > anyone working on this? I understand that this is a nice feature. Should = we > move it to next CF or withdraw CF entry? > > > > At this stage, we should close either Returned With Feedback or > Withdrawn as this requires a lot of work. > > -- > With Regards, > Amit Kapila. > > > > > Hello there, I'm interested in logical DDL replication and I've read through this thread, which has provided me with a lot of valuable information. However, I have a couple of questions that I hope, you could help me with: 1) It seems that a lot of the work done here is simply to extend the existing functionality to work with JSONB. From a development point of view, it seems appropriate to separate this into a new discussion and commit, just to expand the functionality of JSONB in terms of use for development inside the postgres. 2) inside the timeline, it seems that work is moving towards creating an MVP, which can then be finalized. As a result, it seems that the most recent fixes, especially those related to table replication, have been completed, or at least are not discussed in this section. The discussion ends suddenly, so I don't quite understand: are we facing some unsolvable problems, or we just didn't have enough time and energy to bring this to an end? 3) Unfortunately, I couldn't find any specific tests specifically for ddl logical replication. It seems logical to me that covering all possible cases of ddl replication with tests will help move the work forward and convince possible skeptics about worked issues. Did I miss this work or were they just not implemented for some other reason? 4) We decided to use event trigger and logical_message instead of extending the standard logical replication functionality by iterating through WAL records in walsender and decoding there. Was there any reason why we didn't even consider the possibility of doing everything within the structure of the existing logical replication architecture, simply extending it to work with ddl? 5) As for event triggers, I am confused by its use in terms of security and fault tolerance. After reviewing the source code of event triggers, I did not find any problems, but it seems strange that this technology is not used inside Postgres. Perhaps there are reasons for this, or is it such a good technology that it has no problems (or did I just skip this discussion)? 6) Regarding testing: Have any synthetic tests been performed to measure the speed of our replication? How much can we increase the size of WAL, and how does it compare with classical logical and physical replication? -- With Regards, Konstantin Berkaev --00000000000037d085061e0bdd3b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
=D1=87=D1=82, 25 =D0=B8=D1=8E=D0=BB. = 2024=E2=80=AF=D0=B3. =D0=B2 12:35, Amit Kapila <amit.kapila16@gmail.com>:
On Thu, Mar 28, 2024 at 5:31=E2=80=AFPM= Andrey M. Borodin <x4mmm@yandex-team.ru> wrote:
>
> This thread is registered on CF [0] but is not active since 2023. Is a= nyone working on this? I understand that this is a nice feature. Should we = move it to next CF or withdraw CF entry?
>

At this stage, we should close either Returned With Feedback or
Withdrawn as this requires a lot of work.

--
With Regards,
Amit Kapila.





Hello there,
I'm interested in logic= al DDL replication and I've read through this thread, which has provide= d me with a lot of valuable information. However, I have a couple of questi= ons that I hope, you could help me with:
1) It seems that a lot of the w= ork done here is simply to extend the existing functionality to work with J= SONB. From a development point of view, it seems appropriate to separate th= is into a new discussion and commit, just to expand the functionality of JS= ONB in terms of use for development inside the postgres.
2) inside the t= imeline, it seems that work is moving towards creating an MVP, which can th= en be finalized. As a result, it seems that the most recent fixes, especial= ly those related to table replication, have been completed, or at least are= not discussed in this section. The discussion ends suddenly, so I don'= t quite understand: are we facing some unsolvable problems, or we just didn= 't have enough time and energy to bring this to an end?
3) Unfortuna= tely, I couldn't find any specific tests specifically for ddl logical r= eplication. It seems logical to me that covering all possible cases of ddl = replication with tests will help move the work forward and convince possibl= e skeptics about worked issues. Did I miss this work or were they just not = implemented for some other reason?
4) We decided to use event trigger an= d logical_message instead of extending the standard logical replication fun= ctionality by iterating through WAL records in walsender and decoding there= . Was there any reason why we didn't even consider the possibility of d= oing everything within the structure of the existing logical replication ar= chitecture, simply extending it to work with ddl?
5) As for event trigge= rs, I am confused by its use in terms of security and fault tolerance. Afte= r reviewing the source code of event triggers, I did not find any problems,= but it seems strange that this technology is not used inside Postgres. Per= haps there are reasons for this, or is it such a good technology that it ha= s no problems (or did I just skip this discussion)?
6) Regarding te= sting: Have any synthetic tests been performed to measure the speed of our = replication? How much can we increase the size of WAL, and how does it comp= are with classical logical and physical replication?=C2=A0

--
Wit= h Regards,
Konstantin Berkaev
--00000000000037d085061e0bdd3b--