X-Antivirus-mail.weatherbeeta.com.au-Mail-From: 
 phillip.smith@weatherbeeta.com.au via mail.weatherbeeta.com.au
X-Antivirus-mail.weatherbeeta.com.au: 1.25-st-qms (Clear:RC:1(172.20.0.155):.
	Processed in 0.402464 secs Process 14780)
Received: from it-phil.weatherbeeta.com.au (HELO wbaus090) (172.20.0.155)
	by mail.weatherbeeta.com.au with SMTP; 15 Apr 2008 09:26:15 +1000
From: "Phillip Smith" <phillip.smith@weatherbeeta.com.au>
To: "'Ryan Wells'" <ryan.wells@soapware.com>, "'Tom Lane'" <tgl@sss.pgh.pa.us>
Cc: <pgsql-admin@postgresql.org>
Subject: Re: Slow pg_dump 
Date: Tue, 15 Apr 2008 09:26:10 +1000
Organization: Weatherbeeta Pty Ptd
Message-ID: <004b01c89e86$ebe8fbe0$9b0014ac@wbaus090>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.6626
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962
Importance: Normal
In-Reply-To: <EE6D03C0EF73D14E8034C37CA9B627740288D8@exchange.DOCS.COM>
X-Scanned-By-wbeetaau1: Virus scan performed by network-box
X-Scanned-By-wbeetaau1: Scanner file id is wbeetaau1-1208215578.291-14159-000
X-Scanned-By-wbeetaau1: No known viruses found in message (received+scanned in
	0.01/0.03 secs)
X-Scanned-By-wbeetaau1: Spam-Check-Result: No,
	hits=0 required=7 tests= autolearn=no version=2.0
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=0.165 tagged_above=0 required=5 tests=AWL=0.065,
	RDNS_NONE=0.1
X-Spam-Level: 
X-Archive-Number: 200804/131
X-Sequence-Number: 28895

(Sorry, hit send too soon!)

> I ran pg_dump with the arguments you suggested, and my 4 GB test
> table finished backing up in about 25 minutes, which seems great.
> The only problem is that the resulting backup file was over 9 GB.
> Using -Z2 resulting in a 55 minute 6GB backup.
> 
> Here's my interpretation of those results: the TOAST tables for
> our image files are compressed by Postgres.  During the backup,
> pg_dump uncompresses them, and if compression is turned on,
> recompresses the backup.  Please correct me if I'm wrong there.
> 
> If we can't find a workable balance using pg_dump, then it looks
> like our next best alternative may be a utility to handle
> filesystem backups, which is a little scary for on-site,
> user-controlled servers.

How about a post-backup compress?
	pg_dump -Z0 > uncompressed-backup.sql
	gzip uncompressed-backup.sql
	mv uncompressed-backup.sql.gz compressed-backup.sql.gz

Your backup is completed in reasonable time, you're just handling
the storage of the backup afterwards, while users can be using the
System again...


THINK BEFORE YOU PRINT - Save paper if you don't really need to print this

*******************Confidentiality and Privilege Notice*******************

The material contained in this message is privileged and confidential to
the addressee.  If you are not the addressee indicated in this message or
responsible for delivery of the message to such person, you may not copy
or deliver this message to anyone, and you should destroy it and kindly
notify the sender by reply email.

Information in this message that does not relate to the official business
of Weatherbeeta must be treated as neither given nor endorsed by Weatherbeeta.
Weatherbeeta, its employees, contractors or associates shall not be liable
for direct, indirect or consequential loss arising from transmission of this
message or any attachments
e-mail.