From: Evan Bauer <evanbauer@mac.com>
To: Ron <ronljohnsonjr@gmail.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: More efficient pg_restore method?
Date: Tue, 28 Aug 2018 13:13:08 -0400
Message-ID: <931DC3D5-F530-4626-8191-52780D974833@mac.com> (raw)
In-Reply-To: <f6ef298d-fe2f-384d-6be7-0215c579bc42@gmail.com>
References: <44555188-c787-b441-5c69-c56f72905bf4@gmail.com>
<1F9927D1-5C55-48B9-8C6D-ECE3F6BDA3ED@mac.com>
<f6ef298d-fe2f-384d-6be7-0215c579bc42@gmail.com>
Ron,
Even with the speed of your WAN circuit, I would try to do all restores locally. The latency on a WAN connection (especially with all of the files in a format=directory backup) is a concern. The outline of how I would do it are:
rsync or ftp the files to either the database server’s local storage or to or LAN-connected SAN/NAS storage
Verify the copy with a checksum
Perform the pg_restore locally
Cheers,
- Evan
Evan Bauer
eb@evanbauer.com
+1 646 641 2973
Skype: evanbauer
> On Aug 28, 2018, at 13:00, Ron <ronljohnsonjr@gmail.com> wrote:
>
> On 08/28/2018 11:57 AM, Evan Bauer wrote:
>> Ron,
>>
>> A couple of starting questions:
>>
>> What is the size and latency of the network pipe between the primary and backup servers?
>
> 10Gb WAN. Don't know the latency.
>
>> What is the size of the database you need to restore?
>
> 6.5TB dump directory, 3.0TB data/base directory.
>
>> Is there a reason not to do a network copy of the backup directory contents to the database server and run the pg_restore locally?
>
> Option #3!!! I'll research that.
>
>>
>> Cheers,
>>
>> - Evan
>>
>> Evan Bauer
>> eb@evanbauer.com <mailto:eb@evanbauer.com>
>> +1 646 641 2973
>> Skype: evanbauer
>>
>>
>>> On Aug 28, 2018, at 12:48, Ron <ronljohnsonjr@gmail.com <mailto:ronljohnsonjr@gmail.com>> wrote:
>>>
>>>
>>> Pg 9.6.9 on Linux...
>>>
>>> Given a backup server storing a "format=directory" database backup, and a database server, should I:
>>>
>>> Option #1: run pg_restore on the backup server and "push" the data to the database server via port 5432, or
>>> Option #2: have the backup server serve the dump directory via NFS, and run pg_restore on the database server, pulling the data via nfs protocol?
>>>
>>> (It'll be a multi-threaded restore over a 10Gb pipe.)
>>>
>>> --
>>> Angular momentum makes the world go 'round.
>>>
>>
>
> --
> Angular momentum makes the world go 'round.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: evanbauer@mac.com, ronljohnsonjr@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: More efficient pg_restore method?
In-Reply-To: <931DC3D5-F530-4626-8191-52780D974833@mac.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox