Reply-To: jesper.pedersen@redhat.com
Subject: Re: Index Skip Scan
To: James Coleman <jtc331@gmail.com>
Cc: Floris Van Nee <florisvannee@optiver.com>,
 Dmitry Dolgov <9erthalion6@gmail.com>,
 Rafia Sabih <rafia.pghackers@gmail.com>, Jeff Janes <jeff.janes@gmail.com>,
 Thomas Munro <thomas.munro@gmail.com>,
 Alexander Kuzmenkov <a.kuzmenkov@postgrespro.ru>,
 Peter Geoghegan <pg@bowt.ie>, Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 Thomas Munro <thomas.munro@enterprisedb.com>,
 Bhushan Uparkar <bhushan.uparkar@gmail.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Alexander Korotkov <a.korotkov@postgrespro.ru>
References: <2685f020-f302-9bf6-1b5c-ebc2bcebc635@2ndquadrant.com>
 <CA+q6zcUSuFBhGVFZN_AVSxRbt5wr_4_YEYwv8PcQB=m6J6Zpvg@mail.gmail.com>
 <20190315.125452.252656146.horiguchi.kyotaro@lab.ntt.co.jp>
 <CA+q6zcXcSftAxfwuK5c186E8ckZeWmOO07GFvpoOx-wpsDGGzA@mail.gmail.com>
 <CA+q6zcVn10QzbSh686PaZpxgxMhxABRj7aJJpsJ=tKkmcSrBew@mail.gmail.com>
 <CA+q6zcUVAnfSOaSk3s9Sbj6yC7LrDYChSyJd0t7u6Q=97k5a=A@mail.gmail.com>
 <CA+q6zcVQG0ufA_5m_7eeHLN2VRy+z8Kf9FoZ2NnF=yd0_43ndg@mail.gmail.com>
 <CA+q6zcXxwU1FZ=Tj0kWeTkrJ1N9=fhrUH1t5N_yHEcKsG=+qUw@mail.gmail.com>
 <1559361698401.22864@Optiver.com> <1559362223126.18661@Optiver.com>
 <CA+FpmFeKh9BAhDTAf-ZWQ=pvVaRy+Lnahb_8=OOFqcVXJDBWWQ@mail.gmail.com>
 <988b7763-5c55-1be1-b7b6-f3a68eff0d40@redhat.com>
 <CA+q6zcVnRRqktw4cg2Vddy3S+VY3Tsmit0kausYtctvsTum7tA@mail.gmail.com>
 <1559763453006.85615@Optiver.com> <1559763570329.85227@Optiver.com>
 <4d469bf2-49ff-bc3d-a428-6c7b5d87d526@redhat.com>
 <CAAaqYe-kZwjA3bWWOVC_GRyo6yzrrK=YoNM36n2un4oh+TYBpA@mail.gmail.com>
From: Jesper Pedersen <jesper.pedersen@redhat.com>
Organization: Red Hat
Message-ID: <3e72646c-cbeb-be43-8f6c-aa7543131e8e@redhat.com>
Date: Fri, 14 Jun 2019 13:19:31 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.7.0
MIME-Version: 1.0
In-Reply-To: 
 <CAAaqYe-kZwjA3bWWOVC_GRyo6yzrrK=YoNM36n2un4oh+TYBpA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi James,

On 6/13/19 11:40 PM, James Coleman wrote:
> I've previously noted upthread (along with several others), that I
> don't see a good reason to limit this new capability to index only
> scans. In addition to the reasons upthread, this also prevents using
> the new feature on physical replicas since index only scans require
> visibility map (IIRC) information that isn't safe to make assumptions
> about on a replica.
> 
> That being said, it strikes me that this likely indicates an existing
> architecture issue. I was discussing the problem at PGCon with Andres
> and Heiki with respect to an index scan variation I've been working on
> myself. In short, it's not clear to me why we want index only scans
> and index scans to be entirely separate nodes, rather than optional
> variations within a broader index scan node. The problem becomes even
> more clear as we continue to add additional variants that lie on
> different axis, since we end up with an ever multiplying number of
> combinations.
> 
> In that discussion no one could remember why it'd been done that way,
> but I'm planning to try to find the relevant threads in the archives
> to see if there's anything in particular blocking combining them.
> 
> I generally dislike gating improvements like this on seemingly
> tangentially related refactors, but I will make the observation that
> adding the skip scan on top of such a refactored index scan node would
> make this a much more obvious and complete win.
>

Thanks for your feedback !

> As I noted to Jesper at PGCon I'm happy to review the code in detail
> also, but likely won't get to it until later this week or next week at
> the earliest.
> 
> Jesper: Is there anything still on your list of things to change about
> the patch? Or would now be a good time to look hard at the code?
> 

It would be valuable to have test cases for your use-cases which works 
now, or should work.

I revived Thomas' patch because it covered our use-cases and saw it as a 
much needed feature.

Thanks again !

Best regards,
  Jesper