pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Manu <manuelreyesbravo@gmail.com>
To: pgsql-general@lists.postgresql.org
Cc: kavya chandren <kavyachandren@gmail.com>
Cc: Adrian Klaver <adrian.klaver@aklaver.com>
Subject: Re: Challenges with huge postgresql deployments in 300+ TBs
Date: Fri, 09 Oct 2026 15:04:58 -0300
Message-ID: <179156909812.373782.3765533734963931820@gmail.com> (raw)
In-Reply-To: <dc1f76cd-3a3a-4866-b858-c7251fed3d6a@aklaver.com>
References: <dc1f76cd-3a3a-4866-b858-c7251fed3d6a@aklaver.com>

Hi Kavya,

> 3. Fixing a standby that lags a lot

Of the operational walls, this is the one I would size before
committing rather than after, so I put together a small harness that
turns it into a number: it loads indexed data, pauses replay to
separate WAL generation from replay, then times pure replay from local
WAL and reports MB/s with a 95% CI and a lag curve.  About 15 minutes
on a couple hundred MB, and it refuses assert-enabled builds so the
timing is real:

  https://github.com/Manuelreyesbravo/standby-replay-ceiling

The numbers are hardware-specific -- one laptop core replays around
190 MB/s here, CPU-bound.  The point is to run it on a subset of your
intended storage and CPU and get your own ceiling before building the
full system, which matches Adrian's start-from-a-test-setup
suggestion; whether one replay core keeps up then depends on how much
WAL your write workload generates.

It measures that wall, it does not move it.  Happy to adjust it if your
setup would test differently.

Regards,
Manu






view thread (15+ messages)

Message-ID: <179156909812.373782.3765533734963931820@gmail.com>
Permalink:  ../179156909812.373782.3765533734963931820@gmail.com/
Also on:    postgresql.org/message-id/179156909812.373782.3765533734963931820@gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: manuelreyesbravo@gmail.com, kavyachandren@gmail.com, adrian.klaver@aklaver.com
  Subject: Re: Challenges with huge postgresql deployments in 300+ TBs
  In-Reply-To: <179156909812.373782.3765533734963931820@gmail.com>
  References: <dc1f76cd-3a3a-4866-b858-c7251fed3d6a@aklaver.com> <179156909812.373782.3765533734963931820@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox