From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Cc: Andreas Karlsson <andreas@proxel.se>
Cc: Ashwin Agrawal <aagrawal@pivotal.io>
Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
Cc: PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Zedstore - compressed in-core columnar storage
Date: Sun, 14 Apr 2019 19:12:33 +0200
Message-ID: <20190414171233.vhukbvlfwnacc37u@development> (raw)
In-Reply-To: <20190414164510.idlavfnqkutj2h3u@alap3.anarazel.de>
References: <CALfoeiuF-m5jg51mJUPm5GN8u396o5sA2AF5N97vTRAEDYac7w@mail.gmail.com>
<fc2d4f99-1ca6-854f-7025-ec3db94f6e04@catalyst.net.nz>
<f301066e-08e1-a4b6-e058-0c1f24a1cc92@catalyst.net.nz>
<6A2B85A8-1ABD-4F85-981E-C94172D662FD@pivotal.io>
<44cafbdf-0a04-64ef-3cf8-6d5b14b68643@postgrespro.ru>
<93222d26-2141-acf2-61ef-3127f57e80e6@proxel.se>
<59ef6173-6667-8a02-ff7c-a52c8ce05a4f@postgrespro.ru>
<20190414163618.x2bhtfwdulhah7o5@development>
<20190414164510.idlavfnqkutj2h3u@alap3.anarazel.de>
On Sun, Apr 14, 2019 at 09:45:10AM -0700, Andres Freund wrote:
>Hi,
>
>On 2019-04-14 18:36:18 +0200, Tomas Vondra wrote:
>> I think those comparisons are cute and we did a fair amount of them when
>> considering a drop-in replacement for pglz, but ultimately it might be a
>> bit pointless because:
>>
>> (a) it very much depends on the dataset (one algorithm may work great on
>> one type of data, suck on another)
>>
>> (b) different systems may require different trade-offs (high ingestion
>> rate vs. best compression ratio)
>>
>> (c) decompression speed may be much more important
>>
>> What I'm trying to say is that we shouldn't obsess about picking one
>> particular algorithm too much, because it's entirely pointless. Instead,
>> we should probably design the system to support different compression
>> algorithms, ideally at column level.
>
>I think we still need to pick a default algorithm, and realistically
>that's going to be used by like 95% of the users.
>
True. Do you expect it to be specific to the column store, or should be
set per-instance default (even for regular heap)?
FWIW I think the conclusion from past dev meetings was we're unlikely to
find anything better than lz4. I doubt that changed very much.
regard
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, andres@anarazel.de, k.knizhnik@postgrespro.ru, andreas@proxel.se, aagrawal@pivotal.io, mark.kirkwood@catalyst.net.nz
Subject: Re: Zedstore - compressed in-core columnar storage
In-Reply-To: <20190414171233.vhukbvlfwnacc37u@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox