From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Parallel WAL Archival Options
Date: Sun, 6 Aug 2023 03:20:36 -0500
Message-ID: <a45c5157-e2be-8592-9447-c62d414fa3dc@gmail.com> (raw)
In-Reply-To: <CAFpL5Vx86qhPAiOsUyrK=7hwKTuufbYYPp6kEnDJwvb=ZRphYg@mail.gmail.com>
References: <CAFpL5Vx86qhPAiOsUyrK=7hwKTuufbYYPp6kEnDJwvb=ZRphYg@mail.gmail.com>
On 8/6/23 02:43, Nikhil Shetty wrote:
> Hi Team,
>
> I would like to know which backup&restore tools will be better for
> scenarios where the database is generating around 400 WALs per minute.
If my math is correct, 400x 16MB WAL files per minute is 400*(16*2^20)/60*8
/ 10^6 = 895 MBits *per second*. Plus overhead.
That's about *1Gbit/second*. Definitely nothing to sneeze at.
> We are using wal-g but it is not able to keep pace with the wal
> generation. We increased the upload streams to 256 but no luck
Uploads to a *remote* server?
Does wal-g compress files before sending them across the wire? By how
much? Are you CPU or IO bound by having to compress that much data?
--
Born in Arizona, moved to Babylonia.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: ronljohnsonjr@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: Parallel WAL Archival Options
In-Reply-To: <a45c5157-e2be-8592-9447-c62d414fa3dc@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox