X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 826E111F6002 for ; Sat, 29 Apr 2006 00:21:07 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 62558-02 for ; Sat, 29 Apr 2006 00:20:59 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from oss4u.com (mail.oss4u.com [64.142.76.151]) by postgresql.org (Postfix) with SMTP id 2431A11F602A for ; Sat, 29 Apr 2006 00:20:58 -0300 (ADT) Received: (qmail 8984 invoked from network); 29 Apr 2006 03:19:48 -0000 Received: from tiburon.oss4u.com (192.168.1.4) by gw.oss4u.com (192.168.1.1) with ESMTP; 29 Apr 2006 03:19:48 -0000 From: "Uwe C. Schroeder" To: pgsql-admin@postgresql.org Subject: Re: Backing up large databases Date: Fri, 28 Apr 2006 20:20:56 -0700 User-Agent: KMail/1.8.2 Cc: Steve Burrows References: <44523B85.4020605@jla.com> In-Reply-To: <44523B85.4020605@jla.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200604282020.56811.uwe@oss4u.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200604/346 X-Sequence-Number: 21515 Why don't you set up a second server with identical database and use Slony to replicate the primary one there. You can then do whatever you want on the mirror - dump it on a tape, etc. On Friday 28 April 2006 08:57, Steve Burrows wrote: > I am struggling to find an acceptable way of backing up a PostgreSQL 7.4 > database. > > The database is quite large, currently it occupies about 180GB, divided > into two elements, a set of active tables and a set of archive tables > which are only used for insertions. > > I ran pg_dump -Fc recently, it took 23.5 hours to run, and output a > single file of 126GB. Obviously as the database continues to grow it > will soon be so large that it cannot be pg_dumped within a day. Running > rsync to do a complete fresh copy of the pgsql file structure took 4 > hours, but later that day running another iteration of rsync (which > should have only copied changed files) took 3 hours, and I cannot afford > to have the db down that long. > > Anybody with any ideas? The database is being used as the backend for a > mail server, so it has transactions 24 hours a day but is quieter at > night. I want to be able to back it up or replicate it on a daily basis > with minimum downtime so that the mail backlog doesn't get too large. > Ideally I want the first generation of backup/replica going onto the > same machine as the original because the volume of data is such that any > attempt at network or tape backup of the live files will require too > much downtime, once I've got a backup then I can copy that out to other > NAS or tape at leisure. > > If anyone has experience of safeguarding a similarly large PostgreSQL > database with minimal downtime I'd be delighted to hear.. The machine > is running 2 Xeons, 4GB ram and a half-terabyte RAID10 array on a DELL > PERC scsi subsystem, with a load average of around 0.5 - 0.6, so it's > not exactly overstretched. > > Thanks, > > Steve -- UC -- Open Source Solutions 4U, LLC 1618 Kelly St Phone: +1 707 568 3056 Santa Rosa, CA 95401 Cell: +1 650 302 2405 United States Fax: +1 707 568 6416