From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Teodor Sigaev <teodor@sigaev.ru>
To: Michael Paquier <michael.paquier@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Haribabu Kommi <kommi.haribabu@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Pluggable storage
Date: Fri, 23 Jun 2017 16:07:44 +0200
Message-ID: <9b5728fc-c266-5f71-9a69-7668ad231e05@2ndquadrant.com> (raw)
In-Reply-To: <8b2510f0-ab89-dbf1-5b1c-5d686a5e33b8@sigaev.ru>
References: <20160824182201.GA232997@alvherre.pgsql>
<20161013202642.ksiuyx72gu6wgupf@alvherre.pgsql>
<CAJrrPGfG7BarSFb6KQkOOYVaC6vjQRgG45+J2+TwLHHaFKuwGA@mail.gmail.com>
<CA+TgmoYQN9xtMopr4Db3UFCqpbfd5DJb0j5apSmWtkyuWRz0Xw@mail.gmail.com>
<CAB7nPqR6TorHQ_mXmAiB_cL40Vf2+F6hsue-P9HVGoxVxoHyKA@mail.gmail.com>
<8b2510f0-ab89-dbf1-5b1c-5d686a5e33b8@sigaev.ru>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hi,
On 6/23/17 10:38 AM, Teodor Sigaev wrote:
>> 1. Table AM with a 6-byte TID.
>> 2. Table AM with a custom locator format, which could be TID-like.
>> 3. Table AM with no locators.
>
> Currently TID has its own type in system catalog. Seems, it's possible
> that storage claims type of TID which it uses. Then, AM could claim it
> too, so the core based on that information could solve the question
> about AM-storage compatibility. Storage could also claim that it hasn't
> TID type at all so it couldn't be used with any access method, use case:
> compressed column oriented storage.
>
Isn't the fact that TID is an existing type defined in system catalogs
is a fairly insignificant detail? I mean, we could just as easily define
a new 64-bit locator data type, and use that instead, for example.
The main issue here is that we assume things about the TID contents,
i.e. that it contains page/offset etc. And Bitmap nodes rely on that, to
some extent - e.g. when prefetching data.
>
> As I remeber, only GIN depends on TID format, other indexes use it as
> opaque type. Except, at least, btree and GiST - they believe that
> internal pointers are the same as outer (to heap)
>
I think you're probably right - GIN does compress the posting lists by
exploiting the TID redundancy (that it's page/offset structure), and I
don't think there are other index AMs doing that.
But I'm not sure we can simply rely on that - it's possible people will
try to improve other index types (e.g. by adding similar compression to
other index types). Moreover we now support extensions defining custom
index AMs, and we have no clue what people may do in those.
So this would clearly require some sort of flag for each index AM.
>
> Another dubious part - BitmapScan.
>
It would be really great if you could explain why BitmapScans are
dubious, instead of just labeling them as dubious. (I guess you mean
Bitmap Heap Scans, right?)
I see no conceptual issue with bitmap scans on arbitrary locator types,
as long as there's sufficient locality information encoded in the value.
What I mean by that is that for any two locator values A and B:
(1) if (A < B) then (A is stored before B)
(2) if (A is close to B) then (A is stored close to B)
Without these features it's probably futile to try to do bitmap scans,
because the bitmap would not result in mostly sequential access pattern
and things like prefetch would not be very efficient, I think.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, teodor@sigaev.ru, michael.paquier@gmail.com, robertmhaas@gmail.com, kommi.haribabu@gmail.com, alvherre@2ndquadrant.com
Subject: Re: Pluggable storage
In-Reply-To: <9b5728fc-c266-5f71-9a69-7668ad231e05@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox