Date: Mon, 7 Aug 2023 10:42:31 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Nikhil Shetty <nikhil.dba04@gmail.com>
Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Subject: Re: Parallel WAL Archival Options
Message-ID: <ZNEC18tW/KAyYGlG@tamriel.snowman.net>
References: 
 <CAFpL5Vx86qhPAiOsUyrK=7hwKTuufbYYPp6kEnDJwvb=ZRphYg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="ufWoHdoS5l/spTvc"
Content-Disposition: inline
In-Reply-To: 
 <CAFpL5Vx86qhPAiOsUyrK=7hwKTuufbYYPp6kEnDJwvb=ZRphYg@mail.gmail.com>
User-Agent: Mutt/2.1.4 (2021-12-11)
Archived-At: 
 <https://www.postgresql.org/message-id/ZNEC18tW/KAyYGlG%40tamriel.snowman.net>
Precedence: bulk


--ufWoHdoS5l/spTvc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Nikhil Shetty (nikhil.dba04@gmail.com) wrote:
> I would like to know which backup&restore tools will be better for
> scenarios where the database is generating around 400 WALs per minute.
>=20
> We are using wal-g but it is not able to keep pace with the wal generatio=
n.
> We increased the upload streams to 256 but no luck

pgbackrest also does parallel WAL archiving and is able to handle
large volumes of WAL and we'd love to hear how well it works for you.

Feel free to reach out via a github issue if you have trouble making it
able to keep up.

Do be sure to also test that you're able to actually perform restores in
whatever timeframe you expect to be able to with that kind of WAL
generation.

Thanks,

Stephen

--ufWoHdoS5l/spTvc
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEwf6gbxKhD863zrx/7WyKOINHZFUFAmTRAtQACgkQ7WyKOINH
ZFWmMA/+LERCBNrSgEmU3GtC3z9tnU7jm4soaEtBMJcTgyRgh9Gg3MWEz5z8YBLr
g3yVrc91xsHSAO8BuqGk55FFS5pedgBJUuC0+CKpZbZVa7gu3ybP7xUAJQZOIJgA
N+16Jw8GAMgmOBuXgbq/kB65mS/CZ8VwD9og6VFckhPYhYa1SnwakR1z9u4F+Ug6
xfM7SJ6p/2wwvM6zwot75ec4mwqpbeGEDiEy35bFL0UDbNRZ3J0Z16nAXaG/plP6
79ve7YVpiKZsABQYW0uuKTZostRr95xX+wo8XVQvjNkJB67gKzviGf+/XUHzmR3F
N3GL6lcNIQPDHHLneILvxaXL30pu9L6t30rdRtXB2TROiPhuo5Ngbjf4ETphSnlI
l+bmzK8uVZkrag894FKtofUn2CFjgtlA5b4e1KBWryT/j+z4Ou1gPE6EIfjMMjXy
DY+z6oC7273xLf6jNw2BZUaSVNQ799cvAV4uod1luls5U2iubTKYPUJMdrW5h0Qm
6Ot3RUQLe/xhkBaJaErCmgZONl3eiAu/ZtPxHBXe9cRIOHUzHPgLDQ6xyPNzLCt5
jeOVhHaHRhTqwfQquU20zUt7brOkwWLJ1A8fuRW9LvQelLGF9NuMBsZVm718Xsz3
2YnaElWAe6yMvhAgfF4qFT5+UAaca10fF8CYnoj2uhDEG1mFrTY=
=PKN6
-----END PGP SIGNATURE-----

--ufWoHdoS5l/spTvc--