From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Joseph Hammerman <joe.hammerman@datadoghq.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Logical replication and pg_dump for out of band synchronization
Date: Wed, 6 Jul 2022 11:21:31 +0200
Message-ID: <4b3b8dae-fcd6-eb0f-c29f-f27a98a8d5b1@enterprisedb.com> (raw)
In-Reply-To: <CAHs7QM-myW7FiR=5HqAmS2NjpP7L=xjGHM8ELjcDyPNthpNQew@mail.gmail.com>
References: <CAHs7QM-myW7FiR=5HqAmS2NjpP7L=xjGHM8ELjcDyPNthpNQew@mail.gmail.com>
On 04.07.22 22:40, Joseph Hammerman wrote:
> We have been trying to use Logical Replication to synchronize some
> oversized tables (1Tb+).
> We are replicating from 9.6 -> 11.x. The long sync times for the initial
> snapshots of these large tables have been causing incidents however,
> since autovacuum cannot clean up anything older than the xmin horizon.
What replication system are you using with PG 9.6? If you are using
pglogical, then it contains a program pglogical_create_subscriber that
addresses this.
> We then intend to take a dump, restore the table, and play back
> the changes by enabling the Subscription. This way the bulk data
> transfer is out of band from the production application. Our testing
> shows that this works cleanly, and that new changes replicate correctly
> to the target relations. Additionally, pg_dump has a —snapshot flag
> which it appears was added to support this sort of workflow.
Yes, this would also be a valid solution.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: peter.eisentraut@enterprisedb.com, joe.hammerman@datadoghq.com, pgsql-admin@lists.postgresql.org
Subject: Re: Logical replication and pg_dump for out of band synchronization
In-Reply-To: <4b3b8dae-fcd6-eb0f-c29f-f27a98a8d5b1@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox