pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Robert Haas <robertmhaas@gmail.com>
Cc: ktm@rice.edu <ktm@rice.edu>
Cc: Florian Pflug <fgp@phlo.org>
Cc: Dimitri Fontaine <dimitri@2ndquadrant.fr>
Cc: Magnus Hagander <magnus@hagander.net>
Cc: Euler Taveira <euler@timbira.com>
Cc: Marko Kreen <markokr@gmail.com>
Cc: Martijn van Oosterhout <kleptog@svana.org>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Cc: Claes Jakobsson <claes@versed.se>
Subject: Re: libpq compression
Date: Tue, 26 Jun 2012 11:47:30 -0400
Message-ID: <7358.1340725650@sss.pgh.pa.us> (raw)
In-Reply-To: <CA+TgmoYAuMQS1BHF5v1L9Aofh77_9TdQ3bFOWMHLQ4CoY_D=7w@mail.gmail.com>
References: <CA+TgmoaTMxcAFMN8d_FXD1S_3keyq3a=Vwa7ka8cnVepy-jqjg@mail.gmail.com>
	<1D8B2B60-F42E-448C-9295-C07FACCF0611@phlo.org>
	<CABUevEybu+v7zcsM8qtOEFdQ95njyK=ERjVbdMAe3c7T2Ut6JA@mail.gmail.com>
	<7760EC03-5B38-4903-AD2D-95F787B1E1ED@phlo.org>
	<CACMqXCLkOpYnK1eLTb_bAqx344C9j=rchTeNXSW=E6+hdXwyPA@mail.gmail.com>
	<4FE49F7C.9020300@timbira.com>
	<CA+Tgmobw1Hhm4ZoS34o4bJBmMtQrnbU17v01+gCMeGmf8e06Fw@mail.gmail.com>
	<CABUevExTHWBAo_9mMUSOieWumQ66WhOao2MpCfdxUgEMYcW0Dw@mail.gmail.com>
	<m2obo719kz.fsf@2ndQuadrant.fr>
	<E52370CE-442F-46CC-BE5F-C7577FFB283F@phlo.org>
	<20120625202523.GN6547@aart.rice.edu>
	<CA+TgmoYAuMQS1BHF5v1L9Aofh77_9TdQ3bFOWMHLQ4CoY_D=7w@mail.gmail.com>

Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Jun 25, 2012 at 4:25 PM, ktm@rice.edu <ktm@rice.edu> wrote:
>> Here is the benchmark list from the Google lz4 page:
>> 
>> Name            Ratio   C.speed D.speed
>> LZ4 (r59)       2.084   330      915
>> LZO 2.05 1x_1   2.038   311      480
>> QuickLZ 1.5 -1  2.233   257      277
>> Snappy 1.0.5    2.024   227      729
>> LZF             2.076   197      465
>> FastLZ          2.030   190      420
>> zlib 1.2.5 -1   2.728    39      195
>> LZ4 HC (r66)    2.712    18     1020
>> zlib 1.2.5 -6   3.095    14      210

>> lz4 absolutely dominates on compression/decompression speed. While fastlz
>> is faster than zlib(-1) on compression, lz4 is almost 2X faster still.

> At the risk of making everyone laugh at me, has anyone tested pglz?

Another point here is that those Google numbers (assuming that they
apply to our use-cases, a point not in evidence) utterly fail to make
the argument that zlib is not the thing to use.  zlib is beating all
the others on compression ratio, often by 50%.  Before making any
comparisons, you have to make some assumptions about what the network
speed is ... and unless it's pretty damn fast relative to your CPU speed
getting the better compression ratio is going to be more attractive than
saving some cycles.

			regards, tom lane



view thread (289+ messages)  latest in thread

Message-ID: <7358.1340725650@sss.pgh.pa.us>
Permalink:  ../7358.1340725650@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/7358.1340725650@sss.pgh.pa.us

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, robertmhaas@gmail.com, ktm@rice.edu, fgp@phlo.org, dimitri@2ndquadrant.fr, magnus@hagander.net, euler@timbira.com, markokr@gmail.com, kleptog@svana.org, bruce@momjian.us, claes@versed.se
  Subject: Re: libpq compression
  In-Reply-To: <7358.1340725650@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox