Subject: Re: POC: GROUP BY optimization
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Teodor Sigaev <teodor@sigaev.ru>,
 Pgsql Hackers <pgsql-hackers@postgresql.org>
References: <7c79e6a5-8597-74e8-0671-1c39d124c9d6@sigaev.ru>
 <97d148b7-f3e7-1c39-07fd-74cbc66f6ec9@sigaev.ru>
 <6d1e0cdb-dde3-f62a-43e2-e90bbd9b0f42@2ndquadrant.com>
 <b2c0e7bf-0d38-0042-dbbe-7e50b6a3454f@sigaev.ru>
 <7edddb4f-9ddd-561d-2bde-47340f91d2a0@2ndquadrant.com>
 <2cb99655-7ae7-ec8d-2c74-3fea58987104@sigaev.ru>
 <6dddcce6-0d91-1868-794a-4d61dc5516f0@2ndquadrant.com>
 <a02225c6-96d3-2e85-9907-9459105d1244@sigaev.ru>
 <e33558a8-6528-f0c4-8f59-04ff7e2e529c@2ndquadrant.com>
Message-ID: <523d1f60-c60c-5c8c-8cc5-c6c078970da6@2ndquadrant.com>
Date: Sun, 10 Jun 2018 01:18:17 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.2
MIME-Version: 1.0
In-Reply-To: <e33558a8-6528-f0c4-8f59-04ff7e2e529c@2ndquadrant.com>
Content-Type: text/plain; charset=koi8-r
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 06/09/2018 08:09 PM, Tomas Vondra wrote:
> 
> /snip/
> 
> 4) when adding Sort for grouping, try producing the right output order
>    (if the ORDER BY was specified)
> 

BTW I've just realized we already do something similar in master. If you
run a query like this:

  SELECT a, b, count(*) FROM t GROUP BY b, a ORDER BY a;

we will actually plan it like this:

          QUERY PLAN
  ---------------------------
   GroupAggregate
     Group Key: a, b
     ->  Sort
           Sort Key: a, b
           ->  Seq Scan on t
  (5 rows)

I.e. we already do reorder the group clauses to match ORDER BY, to only
require a single sort. This happens in preprocess_groupclause(), which
also explains the reasoning behind that.

I wonder if some of the new code reordering group pathkeys could/should
be moved here (not sure, maybe it's too early for those decisions). In
any case, it might be appropriate to update some of the comments before
preprocess_groupclause() which claim we don't do certain things added by
the proposed patches.

This probably also somewhat refutes my claim that the order of grouping
keys is currently fully determined by users (and so they may pick the
most efficient order), while the reorder-by-ndistinct patch would make
that impossible. Apparently when there's ORDER BY, we already mess with
the order of group clauses - there are ways to get around it (subquery
with OFFSET 0) but it's much less clear.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services