From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Heikki Linnakangas <hlinnaka@iki.fi>
To: Amit Kapila <amit.kapila16@gmail.com>
To: Ashwin Agrawal <aagrawal@pivotal.io>
Cc: PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Zedstore - compressed in-core columnar storage
Date: Wed, 10 Apr 2019 10:38:13 +0300
Message-ID: <b522e7a7-e7f4-47f1-7430-c594f3f712b6@postgrespro.ru> (raw)
In-Reply-To: <7ab5be5f-6404-b52b-e6bf-c1c891ceabb1@iki.fi>
References: <CALfoeiuF-m5jg51mJUPm5GN8u396o5sA2AF5N97vTRAEDYac7w@mail.gmail.com>
<CAA4eK1Kw1h6SE9dO243T8cvak4CPQcvEXpJMPBzkGdVwRP_tBg@mail.gmail.com>
<7ab5be5f-6404-b52b-e6bf-c1c891ceabb1@iki.fi>
On 10.04.2019 10:25, Heikki Linnakangas wrote:
> On 10/04/2019 09:29, Amit Kapila wrote:
>> On Tue, Apr 9, 2019 at 5:57 AM Ashwin Agrawal <aagrawal@pivotal.io>
>> wrote:
>>> Row store
>>> ---------
>>>
>>> The tuples are stored one after another, sorted by TID. For each
>>> tuple, we store its 48-bit TID, a undo record pointer, and the actual
>>> tuple data uncompressed.
>>>
>>
>> Storing undo record pointer with each tuple can take quite a lot of
>> space in cases where you can't compress them.
>
> Yeah. This does depend on compression to eliminate the unused fields
> quite heavily at the moment. But you could have a flag in the header
> to indicate "no undo pointer needed", and just leave it out, when it's
> needed.
>
>> Have you thought how will you implement the multi-locker scheme in
>> this design? In zheap, we have used undo for the same and it is easy
>> to imagine when you have separate transaction slots for each
>> transaction. I am not sure how will you implement the same here.
> I've been thinking that the undo record would store all the XIDs
> involved. So if there are multiple lockers, the UNDO record would
> store a list of XIDs. Alternatively, I suppose you could store
> multiple UNDO pointers for the same tuple.
>
> - Heikki
>
>
I also a little bit confused about UNDO records and MVCC support in
Zedstore. Actually columnar store is mostly needed for analytic for
read-only or append-only data. One of the disadvantages of Postgres is
quite larger per-record space overhead caused by MVCC.
It may be critical if you want to store huge timeseries with relatively
small number of columns (like measurements of some sensor).
It will be nice to have storage format which reduce this overhead when
it is not needed (data is not updated).
Right now, even without UNFO pages, size of zedstore is larger than size
of original Postgres table.
It seems to be very strange.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: k.knizhnik@postgrespro.ru, hlinnaka@iki.fi, amit.kapila16@gmail.com, aagrawal@pivotal.io
Subject: Re: Zedstore - compressed in-core columnar storage
In-Reply-To: <b522e7a7-e7f4-47f1-7430-c594f3f712b6@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox