pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Steve Burrows <steve@jla.com>
To: pgsql-admin@postgresql.org
Subject: Re: Backing up large databases
Date: Thu, 11 May 2006 09:57:18 +0000
Message-ID: <44630A7E.90104@jla.com> (raw)
In-Reply-To: <44523B85.4020605@jla.com>
References: <44523B85.4020605@jla.com>

Just to say thanks to all for your constructive and helpful replies - I 
will build a new server with 8.1 and slony.

Steve


Steve Burrows wrote:
> I am struggling to find an acceptable way of backing up a PostgreSQL 
> 7.4 database.
>
> The database is quite large, currently it occupies about 180GB, 
> divided into two elements, a set of active tables and a set of archive 
> tables which are only used for insertions.
>
> I ran pg_dump -Fc recently, it took 23.5 hours to run, and output a 
> single file of 126GB. Obviously as the database continues to grow it 
> will soon be so large that it cannot be pg_dumped within a day. 
> Running rsync to do a complete fresh copy of the pgsql file structure 
> took 4 hours, but later that day running another iteration of rsync 
> (which should have only copied changed files) took 3 hours, and I 
> cannot afford to have the db down that long.
>
> Anybody with any ideas? The database is being used as the backend for 
> a mail server, so it has transactions 24 hours a day but is quieter at 
> night. I want to be able to back it up or replicate it on a daily 
> basis with minimum downtime so that the mail backlog doesn't get too 
> large. Ideally I want the first generation of backup/replica going 
> onto the same machine as the original because the volume of data is 
> such that any attempt at network or tape backup of the live files will 
> require too much downtime, once I've got a backup then I can copy that 
> out to other NAS or tape at leisure.
>
> If anyone has experience of safeguarding a similarly large PostgreSQL 
> database with minimal downtime I'd be delighted to hear..  The machine 
> is running 2 Xeons, 4GB ram and a half-terabyte RAID10 array on a DELL 
> PERC scsi subsystem, with a load average of around 0.5 - 0.6, so it's 
> not exactly overstretched.
>
> Thanks,
>
> Steve

view thread (10+ messages)

Message-ID: <44630A7E.90104@jla.com>
Permalink:  ../44630A7E.90104@jla.com/
Also on:    postgresql.org/message-id/44630A7E.90104@jla.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: steve@jla.com
  Subject: Re: Backing up large databases
  In-Reply-To: <44630A7E.90104@jla.com>
  References: <44523B85.4020605@jla.com> <44630A7E.90104@jla.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox