pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedToasted column values during replication. (Enhancement)
2+ messages / 2 participants
[nested] [flat]
* Toasted column values during replication. (Enhancement)
@ 2023-06-30 17:10 Chandy G <vgchandru@yahoo.com>
2023-07-03 08:50 ` Re: Toasted column values during replication. (Enhancement) Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 2+ messages in thread
From: Chandy G @ 2023-06-30 17:10 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
We are running into the postgres toasted values showing up in replication slots. We are using debezium and have gone through the following
1. Various options for handling toasted values during replication at the application level.2. Postgres option to turn on replica identitiy to full.
While these are solutions to the problem, each of them comes with their own downside of
(a) custom implementation needed for each consumer. (b) replica identity to full - wal log file size increase. worst case: all tables have columns have toastable columns.
Wanted to understand if there is any scope for enhancement / improvements in postgres itself to handle such scenarios.
Enhancement request: Can the postgres internal component (wal-sender / *) interpret this special toasted-col-value sitting in the wal-file -> replace it with the actual value and send it to the replication slot. I presume, today the wal-sender does not have this support and it just sends whatever data that the wal log has. So if the wal-sender - can intercept such toasted values and translate it to actual values before being sent - that would solve the problem without every consumer having to implement a solution to handle toasted value. This would also help prevent the unwarranted increase of the wal-log if replica-identity to full is set as a w/a.
Thanks.
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Toasted column values during replication. (Enhancement)
2023-06-30 17:10 Toasted column values during replication. (Enhancement) Chandy G <vgchandru@yahoo.com>
@ 2023-07-03 08:50 ` Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 2+ messages in thread
From: Amit Kapila @ 2023-07-03 08:50 UTC (permalink / raw)
To: Chandy G <vgchandru@yahoo.com>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
On Fri, Jun 30, 2023 at 10:40 PM Chandy G <vgchandru@yahoo.com> wrote:
>
> We are running into the postgres toasted values showing up in replication slots. We are using debezium and have gone through the following
>
> 1. Various options for handling toasted values during replication at the application level.
> 2. Postgres option to turn on replica identitiy to full.
>
> While these are solutions to the problem, each of them comes with their own downside of
> (a) custom implementation needed for each consumer.
> (b) replica identity to full - wal log file size increase. worst case: all tables have columns have toastable columns.
>
Which operation are you referring here (insert/update/delete)? It
seems for changed toasted values, we already construct a tuple by
retrieving toast chunks. Are you referring to unchanged toasted
values?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2023-07-03 08:50 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-06-30 17:10 Toasted column values during replication. (Enhancement) Chandy G <vgchandru@yahoo.com>
2023-07-03 08:50 ` Amit Kapila <amit.kapila16@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox