From: Dmitry Dolgov <9erthalion6@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Jesper Pedersen <jesper.pedersen@redhat.com>
Cc: Floris Van Nee <florisvannee@optiver.com>
Cc: David Rowley <david.rowley@2ndquadrant.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: James Coleman <jtc331@gmail.com>
Cc: Rafia Sabih <rafia.pghackers@gmail.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: Bhushan Uparkar <bhushan.uparkar@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Alexander Korotkov <a.korotkov@postgrespro.ru>
Subject: Re: Index Skip Scan
Date: Tue, 21 Jan 2020 11:00:08 +0100
Message-ID: <20200121100008.bsrrp3rtsfvv3ntq@localhost> (raw)
In-Reply-To: <CAH2-WznfCX_XQWWVh+FWbB77aNWTa3MnBzGYffMU0TYYsVBOjg@mail.gmail.com>
References: <1e59d800b38141c5af2ad3a64cf9758e@opammb0562.comp.optiver.com>
<802e46fa-dc2c-4c32-e55b-884e04aa7f12@redhat.com>
<CAH2-Wzk6hvXvZ+1q4JBRunZkxmJwdXxnNsWuKJBp+F+mJuZYZw@mail.gmail.com>
<CAH2-WznfCX_XQWWVh+FWbB77aNWTa3MnBzGYffMU0TYYsVBOjg@mail.gmail.com>
> On Mon, Jan 20, 2020 at 05:05:33PM -0800, Peter Geoghegan wrote:
>
> I suggest that you find a way to add assertions to code like
> _bt_readpage() that verify that we do in fact have the buffer content
> lock. Actually, there is an existing assertion here that covers the
> pin, but not the buffer content lock:
>
> static bool
> _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum)
> {
> <declare variables>
> ...
>
> /*
> * We must have the buffer pinned and locked, but the usual macro can't be
> * used here; this function is what makes it good for currPos.
> */
> Assert(BufferIsValid(so->currPos.buf));
>
> You can add another assertion that calls a new utility function in
> bufmgr.c. That can use the same logic as this existing assertion in
> FlushOneBuffer():
>
> Assert(LWLockHeldByMe(BufferDescriptorGetContentLock(bufHdr)));
>
> We haven't needed assertions like this so far because it's usually it
> is clear whether or not a buffer lock is held (plus the bufmgr.c
> assertions help on their own). The fact that it isn't clear whether or
> not a buffer lock will be held by caller here suggests a problem. Even
> still, having some guard rails in the form of these assertions could
> be helpful. Also, it seems like _bt_scankey_within_page() should have
> a similar set of assertions.
Thanks for suggestion. Agree, we will add such guards. It seems that in
general I need to go through the locking in the patch one more time,
since there are some gaps I din't notice/didn't know about before.
> BTW, there is a paper that describes optimizations like loose index
> scan and skip scan together, in fairly general terms: "Efficient
> Search of Multidimensional B-Trees". Loose index scans are given the
> name "MDAM duplicate elimination" in the paper. See:
>
> http://vldb.org/conf/1995/P710.PDF
>
> Goetz Graefe told me about the paper. It seems like the closest thing
> that exists to a taxonomy or conceptual framework for these
> techniques.
Yes, I've read this paper, as it's indeed the only reference I found
about this topic in literature. But unfortunately it's not much and (at
least from the first read) gives only an overview of the idea.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: 9erthalion6@gmail.com, pg@bowt.ie, jesper.pedersen@redhat.com, florisvannee@optiver.com, david.rowley@2ndquadrant.com, tomas.vondra@2ndquadrant.com, horikyota.ntt@gmail.com, alvherre@2ndquadrant.com, thomas.munro@gmail.com, jtc331@gmail.com, rafia.pghackers@gmail.com, jeff.janes@gmail.com, bhushan.uparkar@gmail.com, a.korotkov@postgrespro.ru
Subject: Re: Index Skip Scan
In-Reply-To: <20200121100008.bsrrp3rtsfvv3ntq@localhost>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox