public inbox for [email protected]  

help / color / mirror / Atom feed

From: Scott Ribe <[email protected]>
To: kasem adel <[email protected]>
Cc: pgsql-admin <[email protected]>
Subject: Re: Advice Needed: Simultaneous Upgrade of Two-Node PostgreSQL 11 Cluster
Date: Tue, 7 Jan 2025 07:10:41 -0700
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAP-pBm-_SNXLaOfPbMxs5WWykwfMdJY6Nmgs21oHo4bJFFxXxg@mail.gmail.com>
References: <CAP-pBm9YJqrZQgrH1ZsFt9RbTsOTKH1niS5bEhhB_mSsT49B5Q@mail.gmail.com>
	<[email protected]>
	<CAP-pBm-Y=b6YxHPag0TLertO=7OCo1kN8RVjBg5mvJNx6Y56KA@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<CAP-pBm-_SNXLaOfPbMxs5WWykwfMdJY6Nmgs21oHo4bJFFxXxg@mail.gmail.com>

> On Jan 7, 2025, at 6:50 AM, kasem adel <[email protected]> wrote:
> 
> Kindly be informed that database size is 1.8 TB and we have an low network bandwidth between two node 10 mbps and to transfer the data from scratch that will take from 21 day to 30 day  for this we need solution in upgrade to prevent load data from scratch .

So, best case, bzip before transfer might get you down to 3 days, maybe. That's one alternative. Others:

- Obviously, if you could get higher network bandwidth temporarily, that would help tremendously.
- Express ship DVD/NVMe/SSD with the backup.
- I assume that if 10Mbps keeps up with changes, most of this data is inactive, are there inactive tables/partitions?
  - Move inactive data into a separate PG cluster, use foreign data wrappers to access it from active so that your users don't see the split.
  - Upgrade cluster with inactive data locally.
  - Whatever the process is to move data from active to inactive, you'll have to suspend it for the duration.
  -  Whether it's a month or 3 days, transfer the base backup of inactive over the slow link (keeping WAL, all the things you have to do to make sure that a replica started against this backup will be in sync and stream when brought up)
  - Proceed with upgrade of cluster with active data (presuming that it is *much* smaller)

Regardless, managing TB+ databases over a 10Mbps link is a challenge.

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Advice Needed: Simultaneous Upgrade of Two-Node PostgreSQL 11 Cluster
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox