From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Cc: Ildar Musin <i.musin@postgrespro.ru>
Subject: Re: [HACKERS] Custom compression methods
Date: Sat, 2 Dec 2017 22:53:10 +0100
Message-ID: <c1b3880a-7a4c-c5bd-7be8-beec0c8684b8@2ndquadrant.com> (raw)
In-Reply-To: <20171202203839.lvbsfyu73igrdmsu@alap3.anarazel.de>
References: <9a0745ab-5b62-d81f-3a6d-ec0929a4a99d@2ndquadrant.com>
<20171130182009.1b492eb2@wp.localdomain>
<9dc0d3ba-5fa2-a7b0-5789-fff3c60cef1c@2ndquadrant.com>
<CA+TgmoZFEhqxXBOONMQeCZ09AJGqE=H2UMySF4zHZsatA0_E2Q@mail.gmail.com>
<a6fe2ee1-7f0e-67d1-7c5d-5075c17191d6@2ndquadrant.com>
<CA+TgmoYGvNV7GqKLLCYcdFEhiANHA_mXxtf_EB=Pd--y14sMQg@mail.gmail.com>
<b6a47ebe-35e3-ad1e-64d9-6e834708ec5a@2ndquadrant.com>
<CA+TgmoZY+TB-wa6QgwsaL-MHCJAhR9n0ifEKTDq8PTxBuqEW-w@mail.gmail.com>
<20171201215217.7ic4izdvj5cishgx@alap3.anarazel.de>
<5cb1995e-a202-2739-bf46-1b6950e5bdb6@2ndquadrant.com>
<20171202203839.lvbsfyu73igrdmsu@alap3.anarazel.de>
On 12/02/2017 09:38 PM, Andres Freund wrote:
> Hi,
>
> On 2017-12-02 16:04:52 +0100, Tomas Vondra wrote:
>> Firstly, it's going to be quite hard (or perhaps impossible) to find an
>> algorithm that is "universally better" than pglz. Some algorithms do
>> work better for text documents, some for binary blobs, etc. I don't
>> think there's a win-win option.
>
> lz4 is pretty much there.
>
That's a matter of opinion, I guess. It's a solid compression algorithm,
that's for sure ...
>> Secondly, all the previous attempts ran into some legal issues, i.e.
>> licensing and/or patents. Maybe the situation changed since then (no
>> idea, haven't looked into that), but in the past the "pluggable"
>> approach was proposed as a way to address this.
>
> Those were pretty bogus.
IANAL so I don't dare to judge on bogusness of such claims. I assume if
we made it optional (e.g. configure/initdb option, it'd be much less of
an issue). Of course, that has disadvantages too (because when you
compile/init with one algorithm, and then find something else would work
better for your data, you have to start from scratch).
>
> I think we're not doing our users a favor if they've to download
> some external projects, then fiddle with things, just to not choose
> a compression algorithm that's been known bad for at least 5+ years.
> If we've a decent algorithm in-core *and* then allow extensibility,
> that's one thing, but keeping the bad and tell forks "please take
> our users with this code we give you" is ...
>
I don't understand what exactly is your issue with external projects,
TBH. I think extensibility is one of the great strengths of Postgres.
It's not all rainbows and unicorns, of course, and it has costs too.
FWIW I don't think pglz is a "known bad" algorithm. Perhaps there are
cases where other algorithms (e.g. lz4) are running circles around it,
particularly when it comes to decompression speed, but I wouldn't say
it's "known bad".
Not sure which forks you're talking about ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, andres@anarazel.de, robertmhaas@gmail.com, i.kurbangaliev@postgrespro.ru, i.musin@postgrespro.ru
Subject: Re: [HACKERS] Custom compression methods
In-Reply-To: <c1b3880a-7a4c-c5bd-7be8-beec0c8684b8@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox