public inbox for [email protected]  

help / color / mirror / Atom feed

From: Paul Smith* <[email protected]>
To: [email protected]
Subject: Re: Guidance Needed: Scaling PostgreSQL for 12 TB Data Growth - New Feature Implementation
Date: Thu, 26 Jun 2025 15:30:22 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAL5Gnis3xKJrUoX-bMP1Sg+E7ptt1hZYMcNN9wcQ48PCt9HNxg@mail.gmail.com>
References: <CAL5Gniu4Gp36jaFOQVXmazMCCUMQMcGD_EvRJTvUx7aSOJcbWA@mail.gmail.com>
	<CANzqJaCju2m=8iD=ijjH-1GiWGwUzrcHEn_2oteRZYdVh_RYJw@mail.gmail.com>
	<CAL5Gnis3xKJrUoX-bMP1Sg+E7ptt1hZYMcNN9wcQ48PCt9HNxg@mail.gmail.com>

On 26/06/2025 14:43, Motog Plus wrote:
> OLTP: This is our primary transactional workload and has replication 
> setup, pgpool - II
> Reporting/DW: This is for reporting purposes.
>
> The growth figures I initially shared (8-9 TB) were a more 
> conservative estimate for OLTP.
>
> However, after a more focused rough estimate for our OLTP workload 
> alone, we anticipate it could reach 35-40 TB of data over the next 5-7 
> years.
>
>
> Specifically for our OLTP databases (which I listed in my initial email):
>
> Database C could reach 30-32 TB, with the acc schema within it 
> potentially growing to 13-15 TB.

The database size is irrelevant (once it's significantly bigger than the 
available RAM)

A Raspberry Pi could easily handle a 30TB database with 50 transactions 
an hour

A 64 core Xeon with 64GB RAM couldn't handle a 500GB database with 
50,000 random insert/update transactions a second.

How many transactions per <time unit> is more important than size, as is 
what sort of transactions are they - eg an indexed SELECT of a single 
row is a lot less effort than an INSERT which has triggers to update 
multiple other tables.

That is what people mean by 'transactional workload'


If the quantity/quality of transactions is staying the same, but, for 
instance, you are simply keeping historical data longer and not querying 
it, then increasing the database size might not be as significant as you 
fear

Paul

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Guidance Needed: Scaling PostgreSQL for 12 TB Data Growth - New Feature Implementation
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox