From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Cc: Ashwin Agrawal <aagrawal@pivotal.io>
Cc: PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Zedstore - compressed in-core columnar storage
Date: Sun, 14 Apr 2019 18:22:10 +0200
Message-ID: <20190414162210.6djspkmbspbd4els@development> (raw)
In-Reply-To: <CA+TgmoardtfM+0nfj=g1qrH5khuHP6Oi5ZS546CLx_iQUG620g@mail.gmail.com>
References: <bf4b8f5f-90c9-025d-013c-87cf998204ad@postgrespro.ru>
<20190409155121.GA4320@alvherre.pgsql>
<CA+TgmoardtfM+0nfj=g1qrH5khuHP6Oi5ZS546CLx_iQUG620g@mail.gmail.com>
On Tue, Apr 09, 2019 at 02:29:09PM -0400, Robert Haas wrote:
>On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>> This is not surprising, considering that columnar store is precisely the
>> reason for starting the work on table AMs.
>>
>> We should certainly look into integrating some sort of columnar storage
>> in mainline. Not sure which of zedstore or VOPS is the best candidate,
>> or maybe we'll have some other proposal. My feeling is that having more
>> than one is not useful; if there are optimizations to one that can be
>> borrowed from the other, let's do that instead of duplicating effort.
>
>I think that conclusion may be premature. There seem to be a bunch of
>different ways of doing columnar storage, so I don't know how we can
>be sure that one size will fit all, or that the first thing we accept
>will be the best thing.
>
>Of course, we probably do not want to accept a ton of storage manager
>implementations is core. I think if people propose implementations
>that are poor quality, or missing important features, or don't have
>significantly different use cases from the ones we've already got,
>it's reasonable to reject those. But I wouldn't be prepared to say
>that if we have two significantly different column store that are both
>awesome code with a complete feature set and significantly disjoint
>use cases, we should reject the second one just because it is also a
>column store. I think that won't get out of control because few
>people will be able to produce really high-quality implementations.
>
>This stuff is hard, which I think is also why we only have 6.5 index
>AMs in core after many, many years. And our standards have gone up
>over the years - not all of those would pass muster if they were
>proposed today.
>
It's not clear to me whether you're arguing for not having any such
implementation in core, or having multiple ones? I think we should aim
to have at least one in-core implementation, even if it's not the best
possible one for all sizes. It's not like our rowstore is the best
possible implementation for all cases either.
I think having a colstore in core is important not just for adoption,
but also for testing and development of the executor / planner bits.
If we have multiple candidates with sufficient code quality, then we may
consider including both. I don't think it's very likely to happen in the
same release, considering how much work it will require. And I have no
idea if zedstore or VOPS are / will be the only candidates - it's way
too early at this point.
FWIW I personally plan to focus primarily on the features that aim to
be included in core, and that applies to colstores too.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, robertmhaas@gmail.com, alvherre@2ndquadrant.com, k.knizhnik@postgrespro.ru, aagrawal@pivotal.io
Subject: Re: Zedstore - compressed in-core columnar storage
In-Reply-To: <20190414162210.6djspkmbspbd4els@development>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox