Message-ID: <4BA62596.8050604@postnewspapers.com.au>
Date: Sun, 21 Mar 2010 21:56:38 +0800
From: Craig Ringer <craig@postnewspapers.com.au>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: David Newall <postgresql@davidnewall.com>
CC: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-performance@postgresql.org,
	robertmhaas@gmail.com, dcrooke@gmail.com
Subject: Re: pg_dump far too slow
References: <4B9C97E1.40509@davidnewall.com> <24696.1268598111@sss.pgh.pa.us>
	<4BA61C74.70506@davidnewall.com>
In-Reply-To: <4BA61C74.70506@davidnewall.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 21/03/2010 9:17 PM, David Newall wrote:
> Thanks for all of the suggestions, guys, which gave me some pointers on
> new directions to look, and I learned some interesting things.
>

> Unfortunately one of these processes dropped eventually, and, according
> to top, the only non-idle process running was gzip (100%.) Obviously
> there were postgress and pg_dump processes, too, but they were throttled
> by gzip's rate of output and effectively idle (less than 1% CPU). That
> is also interesting. The final output from gzip was being produced at
> the rate of about 0.5MB/second, which seems almost unbelievably slow.

CPU isn't the only measure of interest here.

If pg_dump and the postgres backend it's using are doing simple work 
such as reading linear data from disk, they won't show much CPU activity 
even though they might be running full-tilt. They'll be limited by disk 
I/O or other non-CPU resources.

> and wonder if I should read up on gzip to find why it would work so
> slowly on a pure text stream, albeit a representation of PDF which
> intrinsically is fairly compressed.

In fact, PDF uses deflate compression, the same algorithm used for gzip. 
Gzip-compressing PDF is almost completely pointless - all you're doing 
is compressing some of the document structure, not the actual content 
streams. With PDF 1.5 and above using object and xref streams, you might 
not even be doing that, instead only compressing the header and trailer 
dictionary, which are probably in the order of a few hundred bytes.

Compressing PDF documents is generally a waste of time.

--
Craig Ringer