Date: Mon, 16 Sep 2019 12:32:44 +0200
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: James Coleman <jtc331@gmail.com>
Cc: Peter Geoghegan <pg@bowt.ie>, Simon Riggs <simon@2ndquadrant.com>,
	Rafia Sabih <rafia.pghackers@gmail.com>,
	Shaun Thomas <shaun.thomas@2ndquadrant.com>,
	Dmitry Dolgov <9erthalion6@gmail.com>,
	Alexander Korotkov <a.korotkov@postgrespro.ru>,
	Tom Lane <tgl@sss.pgh.pa.us>,
	PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Subject: Re: [PATCH] Incremental sort (was: PoC: Partial sort)
Message-ID: <20190916103244.eqkifumgacmo6hmc@development>
References: <20190708135933.oco3gyloeacdj7yy@development>
 <CAAaqYe_5+as+Mn7MedyVCb3WBXLfhBucEPFHa+g_sN4yXPfBtQ@mail.gmail.com>
 <20190708145853.x2doaqcyj3enol2e@development>
 <CAAaqYe_VMCbq2+mNsMr8NtmDCBWNfHHDHHYppM12JDkudQjPhQ@mail.gmail.com>
 <20190709013703.3mbdhsv2gyxtjc42@development>
 <CAAaqYe8VvKvpZT9V=fuTk-5k=OVQSzqRzY=9dAt3cMz=aiNsLw@mail.gmail.com>
 <20190720132244.3vgg2uynfpxh3me5@development>
 <CAAaqYe-8pxV1UfxbbN7WTXUPhs9g1_swZEomoEp4J0BGpnv_Sw@mail.gmail.com>
 <20190720152512.jpdi2g3notbjcxsd@development>
 <CAAaqYe-u204FcZ8Vqv=qgPfn5mLt3QNpiuZXNc+Kxb5N6w9g1w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: 
 <CAAaqYe-u204FcZ8Vqv=qgPfn5mLt3QNpiuZXNc+Kxb5N6w9g1w@mail.gmail.com>
User-Agent: NeoMutt/20180716
Precedence: bulk

On Sun, Sep 15, 2019 at 09:33:33PM -0400, James Coleman wrote:
>On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> >> ...
>> >>
>> >> I think this may be a thinko, as this plan demonstrates - but I'm not
>> >> sure about it. I wonder if this might be penalizing some other types of
>> >> plans (essentially anything with limit + gather).
>> >>
>> >> Attached is a WIP patch fixing this by considering both startup and
>> >> total cost (by calling compare_path_costs_fuzzily).
>> >
>> >It seems to me that this is likely a bug, and not just a changed
>> >needed for this. Do you think it's better addressed in a separate
>> >thread? Or retain it as part of this patch for now (and possibly break
>> >it out later)? On the other hand, it's entirely possible that someone
>> >more familiar with parallel plan limitations could explain why the
>> >above comment holds true. That makes me lean towards asking in a new
>> >thread.
>> >
>>
>> Maybe. I think creating a separate thread would be useful, provided we
>> manage to demonstrate the issue without an incremental sort.
>
>I did some more thinking about this, and I can't currently come up
>with a way to reproduce this issue outside of this patch. It doesn't
>seem reasonable to me to assume that there's anything inherent about
>this patch that means it's the only way we can end up with a partial
>path with a low startup cost we'd want to prefer.
>
>Part of me wants to pull it over to a separate thread just to get
>additional feedback, but I'm not sure how useful that is given we
>don't currently have an example case outside of this patch.
>

Hmm, I see.

While I initially suggested to start a separate thread only if we have
example not involving an incremental sort, that's probably not a hard
requirement. I think it's fine to start a thead briefly explaining the
issue, and pointing to incremental sort thread for actual example.

>
>One thing to note though: the current patch does not also modify
>add_partial_path_precheck which also does not take into account
>startup cost, so we probably need to update that for completeness's
>sake.
>

Good point. It does indeed seem to make the same assumption about only
comparing total cost before calling add_path_precheck.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services