pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedGZIP of pre-zipped output
3+ messages / 3 participants
[nested] [flat]
* GZIP of pre-zipped output
@ 2010-03-21 17:04 Dave Crooke <dcrooke@gmail.com>
2010-03-22 02:46 ` Re: GZIP of pre-zipped output Craig Ringer <craig@postnewspapers.com.au>
0 siblings, 1 reply; 3+ messages in thread
From: Dave Crooke @ 2010-03-21 17:04 UTC (permalink / raw)
To: David Newall <postgresql@davidnewall.com>; +Cc: pgsql-performance@postgresql.org, robertmhaas@gmail.com, Tom Lane <tgl@sss.pgh.pa.us>; Craig Ringer <craig@postnewspapers.com.au>
If you are really so desparate to save a couple of GB that you are resorting
to -Z9 then I'd suggest using bzip2 instead.
bzip is designed for things like installer images where there will be
massive amounts of downloads, so it uses a ton of cpu during compression,
but usually less than -Z9 and makes a better result.
Cheers
Dave
On Mar 21, 2010 10:50 AM, "David Newall" <postgresql@davidnewall.com> wrote:
Tom Lane wrote:
>
> I would bet that the reason for the slow throughput is that gzip
> is fruitlessl...
Indeed, I didn't expect much reduction in size, but I also didn't expect a
four-order of magnitude increase in run-time (i.e. output at 10MB/second
going down to 500KB/second), particularly as my estimate was based on
gzipping a previously gzipped file. I think it's probably pathological
data, as it were. Might even be of interest to gzip's maintainers.
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: GZIP of pre-zipped output
2010-03-21 17:04 GZIP of pre-zipped output Dave Crooke <dcrooke@gmail.com>
@ 2010-03-22 02:46 ` Craig Ringer <craig@postnewspapers.com.au>
2010-03-22 03:00 ` Re: GZIP of pre-zipped output Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 1 reply; 3+ messages in thread
From: Craig Ringer @ 2010-03-22 02:46 UTC (permalink / raw)
To: Dave Crooke <dcrooke@gmail.com>; +Cc: David Newall <postgresql@davidnewall.com>; pgsql-performance@postgresql.org, robertmhaas@gmail.com, Tom Lane <tgl@sss.pgh.pa.us>
On 22/03/2010 1:04 AM, Dave Crooke wrote:
> If you are really so desparate to save a couple of GB that you are
> resorting to -Z9 then I'd suggest using bzip2 instead.
>
> bzip is designed for things like installer images where there will be
> massive amounts of downloads, so it uses a ton of cpu during
> compression, but usually less than -Z9 and makes a better result.
bzip2 doesn't work very well on gzip'd (deflated) data, though. For good
results, you'd want to feed it uncompressed data, which is a bit of a
pain when the compression is part of the PDF document structure and when
you otherwise want the PDFs to remain compressed.
Anyway, if you're going for extreme compression, these days 7zip is
often a better option than bzip2.
--
Craig Ringer
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: GZIP of pre-zipped output
2010-03-21 17:04 GZIP of pre-zipped output Dave Crooke <dcrooke@gmail.com>
2010-03-22 02:46 ` Re: GZIP of pre-zipped output Craig Ringer <craig@postnewspapers.com.au>
@ 2010-03-22 03:00 ` Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: Scott Marlowe @ 2010-03-22 03:00 UTC (permalink / raw)
To: Craig Ringer <craig@postnewspapers.com.au>; +Cc: Dave Crooke <dcrooke@gmail.com>; David Newall <postgresql@davidnewall.com>; pgsql-performance@postgresql.org, robertmhaas@gmail.com, Tom Lane <tgl@sss.pgh.pa.us>
On Sun, Mar 21, 2010 at 8:46 PM, Craig Ringer
<craig@postnewspapers.com.au> wrote:
> On 22/03/2010 1:04 AM, Dave Crooke wrote:
>>
>> If you are really so desparate to save a couple of GB that you are
>> resorting to -Z9 then I'd suggest using bzip2 instead.
>>
>> bzip is designed for things like installer images where there will be
>> massive amounts of downloads, so it uses a ton of cpu during
>> compression, but usually less than -Z9 and makes a better result.
>
> bzip2 doesn't work very well on gzip'd (deflated) data, though. For good
> results, you'd want to feed it uncompressed data, which is a bit of a pain
> when the compression is part of the PDF document structure and when you
> otherwise want the PDFs to remain compressed.
>
> Anyway, if you're going for extreme compression, these days 7zip is often a
> better option than bzip2.
There's often a choice of two packages, 7z, and 7za, get 7za, it's the
later model version.
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2010-03-22 03:00 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-03-21 17:04 GZIP of pre-zipped output Dave Crooke <dcrooke@gmail.com>
2010-03-22 02:46 ` Craig Ringer <craig@postnewspapers.com.au>
2010-03-22 03:00 ` Scott Marlowe <scott.marlowe@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox