Re: Parallelize WAL Sender and WAL receiver

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Rui DeSousa <[email protected]>
To: Karthik Yellapragada <[email protected]>
Cc: Pgsql-admin <[email protected]>
Subject: Re: Parallelize WAL Sender and WAL receiver
Date: Wed, 4 Sep 2024 21:52:01 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <CACXtxt_ANB20ZWjFQt=BOtCN50g6MvNvQPokv5TwPSwwXchdkw@mail.gmail.com>
References: <CACXtxt_ANB20ZWjFQt=BOtCN50g6MvNvQPokv5TwPSwwXchdkw@mail.gmail.com>

> On Sep 4, 2024, at 9:03 PM, Karthik Yellapragada <[email protected]> wrote:
> 
> Hello all,
> 
> I frequently face the problem of wals generated faster than the wals transferred and applied.
> 
> Is there a way to speed up the process? So I don’t accumulate a lot of WALs at the primary?
> 
> Regards,
> Karthik.

What do you mean by parallelized WAL sender? Do you have many replicas connecting to the primary? Have you tried cascading replication?

What’s the network between the systems? Is a WAN with high latency? Can the replica pull from the WAL file from an archive?  For a high latency networks like New York to London; I would terminate the WAL receiver via a script if replication exceeded an acceptable delay to ensure SLA  where met.  It is faster to pull the WAL files from the local archive which where already replicated to London than streaming replication over a high latency WAN.  After It catches up, it will reconnect to streaming replication.

view thread (10+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Parallelize WAL Sender and WAL receiver
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox