pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-general@lists.postgresql.org <pgsql-general@lists.postgresql.org>
Subject: Re: pg_dump/pg_restore --jobs practical limit?
Date: Fri, 3 Nov 2023 08:19:42 -0500
Message-ID: <413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com> (raw)
In-Reply-To: <CADX_1aanC4gibvwJJeomUxgxTT9xw8gAvRctc4jwgNbSpJO9nQ@mail.gmail.com>
References: <cabaebac-ae7b-493a-a127-aee941f5f335@gmail.com>
	<0c82a8d32181fca3b979347f34ffbb2f7fdf0e90.camel@cybertec.at>
	<93e2708b-2dae-4a04-8660-ba5e1f5b857a@gmail.com>
	<CH0PR10MB74985C1A93DB676D20EF8DDDF7A6A@CH0PR10MB7498.namprd10.prod.outlook.com>
	<1828c313-5a85-4e31-93b4-ff2a37fad862@gmail.com>
	<CADX_1aanC4gibvwJJeomUxgxTT9xw8gAvRctc4jwgNbSpJO9nQ@mail.gmail.com>

On 11/3/23 05:09, Marc Millas wrote:
>
> Marc MILLAS
> Senior Architect
> +33607850334
> www.mokadb.com <http://www.mokadb.com;
>
>
>>     Testing pg_restore with different --jobs= values will be easier.  
>>     pg_dump is what's going to be reading from a constantly varying system.
>     Hello,
>
> each time I do a replatforming of this kind, with DB up to 2 TB, I did 
> create the target DB, eventually needed users then the appropriate 
> databases, and finally,  a simple script to pipe pg_dump into psql, 
> databases one by one.
> So.. one thread. Each time, it was limited by the network bandwidth. My 
> last replatforming with a 10 Gb net and a 1.5 TB DB did show a transfer of 
> 500 Mbytes per second (5Gbs) so.. less than an hour.

I'm shocked that old-school "SQL" single-threading works that fast.

Thanks for the data point.

> which is just fine. Launch it, have lunch, a coffee, and ...done for 
> test.  For Prod, I am used to do it at the quietest night of the week end. 
> and have a nap ( a short one !)...:-)

-- 
Born in Arizona, moved to Babylonia.

view thread (8+ messages)

Message-ID: <413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com>
Permalink:  ../413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com/
Also on:    postgresql.org/message-id/413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: ronljohnsonjr@gmail.com
  Subject: Re: pg_dump/pg_restore --jobs practical limit?
  In-Reply-To: <413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com>
  References: <cabaebac-ae7b-493a-a127-aee941f5f335@gmail.com> <0c82a8d32181fca3b979347f34ffbb2f7fdf0e90.camel@cybertec.at> <93e2708b-2dae-4a04-8660-ba5e1f5b857a@gmail.com> <CH0PR10MB74985C1A93DB676D20EF8DDDF7A6A@CH0PR10MB7498.namprd10.prod.outlook.com> <1828c313-5a85-4e31-93b4-ff2a37fad862@gmail.com> <CADX_1aanC4gibvwJJeomUxgxTT9xw8gAvRctc4jwgNbSpJO9nQ@mail.gmail.com> <413c6c5e-3c1d-47c6-98cd-a7f046493c76@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox