pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedFastest Backup Method?
3+ messages / 3 participants
[nested] [flat]
* Fastest Backup Method?
@ 2017-09-04 17:23 Artem Tomyuk <admin@leboutique.com>
2017-09-05 18:50 ` Re: Fastest Backup Method? Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 1 reply; 3+ messages in thread
From: Artem Tomyuk @ 2017-09-04 17:23 UTC (permalink / raw)
To: pgsql-admin
Hi.
I'am wondering, what is the fastest backup method do you know, for me it is:
(Assume that we running on 16 * CPU box)
pg_dump -Z 4 -j 16 -F c -F d
-Z 4 compression level
-j 16 number of parallel jobs
-F c, d custom+directory format
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Fastest Backup Method?
2017-09-04 17:23 Fastest Backup Method? Artem Tomyuk <admin@leboutique.com>
@ 2017-09-05 18:50 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-09-05 21:12 ` Re: Fastest Backup Method? Greg Spiegelberg <gspiegelberg@gmail.com>
0 siblings, 1 reply; 3+ messages in thread
From: Peter Eisentraut @ 2017-09-05 18:50 UTC (permalink / raw)
To: Artem Tomyuk <admin@leboutique.com>; pgsql-admin
On 9/4/17 13:23, Artem Tomyuk wrote:
> I'am wondering, what is the fastest backup method do you know, for me it is:
> (Assume that we running on 16 * CPU box)
>
> pg_dump -Z 4 -j 16 -F c -F d
>
> -Z 4 compression level
>
> -j 16 number of parallel jobs
>
> -F c, d custom+directory format
I think you misunderstood that last part. There can only be one output
format, and the last option wins. So you are really running in
directory format only.
As for performance, you generally have the right idea. Specifics depend
on the hardware.
My experience is that any compression at all can be bad for performance.
But the I/O cost might balance that in some cases.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Fastest Backup Method?
2017-09-04 17:23 Fastest Backup Method? Artem Tomyuk <admin@leboutique.com>
2017-09-05 18:50 ` Re: Fastest Backup Method? Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
@ 2017-09-05 21:12 ` Greg Spiegelberg <gspiegelberg@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: Greg Spiegelberg @ 2017-09-05 21:12 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: Artem Tomyuk <admin@leboutique.com>; pgsql-admin
On Tue, Sep 5, 2017 at 12:50 PM, Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:
> On 9/4/17 13:23, Artem Tomyuk wrote:
> > I'am wondering, what is the fastest backup method do you know, for me it
> is:
> > (Assume that we running on 16 * CPU box)
> >
> > pg_dump -Z 4 -j 16 -F c -F d
> >
> > -Z 4 compression level
> >
> > -j 16 number of parallel jobs
> >
> > -F c, d custom+directory format
>
> I think you misunderstood that last part. There can only be one output
> format, and the last option wins. So you are really running in
> directory format only.
>
> As for performance, you generally have the right idea. Specifics depend
> on the hardware.
>
> My experience is that any compression at all can be bad for performance.
> But the I/O cost might balance that in some cases.
>
>
In my experience and experimentation, using a hot standby in conjunction
with PITR almost completely insulates the master from backup load. The
recipe is simply obviously configure for WAL archiving, have a hot standby
for backups, execute pg_start_backup(), wait on hot standby for the LSN on
the hot standby, execute pg_xlog_replay_pause() on hot standby, execute
pg_stop_backup() on master, do the necessary backup on hot standby, and
wrap it up with pg_xlog_replay_resume() on hot standby. It's been a while
so that was from memory.
I have performed this test in high transaction environment and tested the
restore with much success. Aside from the checkpoint on the master and
archive_command, minimal to no load incurred on the master. I don't
believe xlog replay pause/resume is strictly necessary but seemed like a
good idea.
HTH,
-Greg
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2017-09-05 21:12 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-09-04 17:23 Fastest Backup Method? Artem Tomyuk <admin@leboutique.com>
2017-09-05 18:50 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-09-05 21:12 ` Greg Spiegelberg <gspiegelberg@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox