MIME-Version: 1.0
References: <CADTnmPjcbW07j26AxdqAsMbSJEAdHtva7Kdazmo99twuF59whw@mail.gmail.com>
In-Reply-To: <CADTnmPjcbW07j26AxdqAsMbSJEAdHtva7Kdazmo99twuF59whw@mail.gmail.com>
From: Akashkiran Shivakumar <akashkiran.foss@gmail.com>
Date: Sat, 9 Aug 2025 20:00:28 -0700
Message-ID: <CADTnmPjxCQMLGMw5+t2qY0inhJoY85u8MNPRGzsZK7fPBevH7Q@mail.gmail.com>
Subject: Re: Can postgres replication slot using pgoutput release multiple CDC
 records for a single update to a particular row
To: pgsql-general@lists.postgresql.org
Content-Type: multipart/alternative; boundary="00000000000005ef8e063bfa03d9"
Archived-At: <https://www.postgresql.org/message-id/CADTnmPjxCQMLGMw5%2Bt2qY0inhJoY85u8MNPRGzsZK7fPBevH7Q%40mail.gmail.com>
Precedence: bulk

--00000000000005ef8e063bfa03d9
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Can someone take a look at this ? Any help would be appreciated

Regards,
Akashkiran

On Tue, Jul 29, 2025 at 3:37=E2=80=AFPM Akashkiran Shivakumar <
akashkiran.foss@gmail.com> wrote:

> Hello,
> I have a postgres database (major version 13) and am doing CDC by using a
> replication slot with pgoutput. In our data lake, we see that there are
> multiple updates (3 in this case) happening to the same row as part of th=
e
> same transaction. This doesn't make sense if we look at them as separate
> updates. The expectation was that the row was updated once and a single C=
DC
> record was pushed out.  I haven't completely ruled out whether multiple
> updates happened in that transaction but I wanted to ask the community if
> it is possible that a single update statement could spill over as multipl=
e
> CDC update records by pgoutput / postgres ?
>
> If yes, could you possibly point to the testcases or code where this migh=
t
> happen? Any blogs or suggestions are welcome. LMK if you need further
> information
>
> P.S: Each update seen in the data lake changes at least one field in the
> row. The entire row with all the columns are pushed out for the 3 updates
> seen in the data lake. From the application perspective, it makes sense i=
f
> the 3 updates were merged into a single update record
>
> Regards,
> Akashkiran
>

--00000000000005ef8e063bfa03d9
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">Can someone take a look at this ? Any hel=
p would=C2=A0be appreciated</div><div><br>Regards,=C2=A0<br>Akashkiran</div=
><br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" clas=
s=3D"gmail_attr">On Tue, Jul 29, 2025 at 3:37=E2=80=AFPM Akashkiran Shivaku=
mar &lt;<a href=3D"mailto:akashkiran.foss@gmail.com">akashkiran.foss@gmail.=
com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"marg=
in:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1e=
x"><div dir=3D"ltr">Hello,=C2=A0<br>I have a postgres database (major versi=
on 13) and am doing CDC by using a replication slot with pgoutput. In our d=
ata lake, we see that there are multiple updates (3 in this case) happening=
 to the same row as part of the same transaction. This doesn&#39;t make=C2=
=A0sense if we look at them as separate updates. The expectation was that t=
he row was updated once and a single CDC record was pushed out.=C2=A0=C2=A0=
I haven&#39;t completely ruled out whether multiple updates happened in tha=
t transaction but I wanted to ask the community if it is possible that a si=
ngle update statement could spill over as multiple CDC update=C2=A0records=
=C2=A0by pgoutput=C2=A0/ postgres=C2=A0?=C2=A0<br><div><br>If yes, could yo=
u possibly point to the=C2=A0testcases or code where this might happen? Any=
 blogs or suggestions are welcome. LMK if you need further information<br><=
br></div><div>P.S: Each update seen in the data lake changes at least one f=
ield in the row. The entire row with all the columns are pushed out for the=
 3 updates seen in the data lake. From the application perspective, it make=
s sense if the 3 updates were merged into a single update record<br><div><b=
r></div><div>Regards,</div><div>Akashkiran</div></div></div>
</blockquote></div></div>

--00000000000005ef8e063bfa03d9--