Message-ID: <5B74101B.9080402@lab.ntt.co.jp>
Date: Wed, 15 Aug 2018 20:35:55 +0900
From: Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:11.0) Gecko/20120327 Thunderbird/11.0.1
MIME-Version: 1.0
Subject: Re: Expression errors with "FOR UPDATE" and postgres_fdw with
 partition
 wise join enabled.
References: <5AFC0865.8050802@lab.ntt.co.jp>
 <CA+TgmobR9XWEhmF3s_WsoLNnnjaFyS_xtbrVG3dCGiUgof20Nw@mail.gmail.com>
 <CA+TgmoZSaKq-fYALn5jf6c_X3==Rb2s8eqLDwGpV=LNNhTXYwg@mail.gmail.com>
 <5B55886F.5090906@lab.ntt.co.jp>
 <CA+TgmoYYTCQL=TUYbUz2b=CsQeB4o1L2+_=zHynxGVmyKUtH1A@mail.gmail.com>
 <5B5712C8.9030404@lab.ntt.co.jp>
 <CA+TgmoazFzdp5ZVwUjk_b2cuB90WCeeWtzt1ShSbXGyg67c+Zw@mail.gmail.com>
 <5B59BA55.30200@lab.ntt.co.jp> <5B619D23.4010000@lab.ntt.co.jp>
 <CA+TgmoZMdNiKUGV0=-QLV1f1-Xr0CDT+8y9wLktuEiQGkt64UQ@mail.gmail.com>
 <CA+TgmoZjYHtkw-o_4=-GXb6obF1epyivr+WnZ1SQCnU6fgtf+w@mail.gmail.com>
 <5358.1533241519@sss.pgh.pa.us> <5B64562A.9000404@lab.ntt.co.jp>
 <5B645868.6030609@lab.ntt.co.jp> <5B683F60.2070806@lab.ntt.co.jp>
 <CA+TgmoYhgEkcR1E-BMS-2G8A4_5z0bbN6vCO+uR7HyQCOsWpEw@mail.gmail.com>
 <5B717A7B.4010308@lab.ntt.co.jp>
 <CA+TgmoaNuWR-Krw9L2NOARNd7gJfHXwgOO8=8J4YAp+P4a1djA@mail.gmail.com>
 <5B739D46.6040000@lab.ntt.co.jp>
 <3585ad1d-d610-ad53-80e2-296050cd73f3@lab.ntt.co.jp>
In-Reply-To: <3585ad1d-d610-ad53-80e2-296050cd73f3@lab.ntt.co.jp>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
To: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Cc: Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
        Andres Freund <andres@anarazel.de>,
        Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>,
        Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Precedence: bulk

(2018/08/15 13:04), Amit Langote wrote:
> On 2018/08/15 12:25, Etsuro Fujita wrote:
>> (2018/08/15 0:51), Robert Haas wrote:
>>> On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita
>>> <fujita.etsuro@lab.ntt.co.jp>   wrote:
>>>> One thing I noticed might be an improvement is to skip
>>>> build_joinrel_partition_info if the given joinrel will be to have
>>>> consider_partitionwise_join=false; in the previous patch, that function
>>>> created the joinrel's partition info such as part_scheme and part_rels if
>>>> the joinrel is considered as partitioned, independently of the flag
>>>> consider_partitionwise_join for it, but if that flag is false, we don't
>>>> generate PWJ paths for the joinrel, so we would not need to create that
>>>> partition info at all.  This would not only avoid unnecessary
>>>> processing in
>>>> that function, but also make unnecessary the changes I made to
>>>> try_partitionwise_join, generate_partitionwise_join_paths,
>>>> apply_scanjoin_target_to_paths, and create_ordinary_grouping_paths.  So I
>>>> updated the patch that way.  Please find attached an updated version of
>>>> the
>>>> patch.
>>>
>>> I guess the question is whether there are (or might be in the future)
>>> other dependencies on part_scheme.  For example, it looks like
>>> partition pruning uses it.  I'm not sure whether partition pruning
>>> supports a plan like:
>>>
>>> Append
>>> ->   Nested Loop
>>>     ->   Seq Scan on p1
>>>     ->   Index Scan on q1
>>> <repeat the above for p2/q2 etc.>
>>
>> I'm not sure that either, but if a join relation doesn't have part_scheme
>> set, it means that that relation is considered as non-partitioned, as in
>> the case when enable_partitionwise_join is off, so there would be no PWJ
>> paths generated for it, to begin with.  So in that case, ISTM that we
>> don't need to worry about that at least for partition pruning.
>
> Fwiw, partition pruning works only for base rels, which applies to both
> planning-time pruning (pruning is performed only during base rel size
> estimation) and run-time pruning (we'll add pruning info to the Append
> plan only if the source AppendPath's parent rel is a base rel).

Thanks for that, Amit!

I looked into the question for the join or upper rel case, but couldn't 
find any places in the PG11 code where we assume that a join or upper 
rel has non-NULL part_scheme, as described below:

* Both try_partitionwise_join() and generate_partitionwise_join_paths() 
check whether a join rel to be considered has non-NULL part_scheme, 
through the IS_PARTITIONED_REL macro:

#define IS_PARTITIONED_REL(rel) \
     ((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
      (rel)->part_rels && !(IS_DUMMY_REL(rel)))

If IS_PARTITIONED_REL, the former adds paths to child-joins, and the 
latter builds PWJ paths; but both don't assume non-NULL part_scheme.

* apply_scanjoin_target_to_paths() checks whether the topmost scan/join 
rel has non-NULL part_scheme, and if so, it recursively adjusts all 
partitions; it doesn't assume non-NULL part_scheme.

* create_ordinary_grouping_paths() also checks whether the topmost 
scan/join rel adjusted by apply_scanjoin_target_to_paths() has non-NULL 
part_scheme, and if so, it considers PWA paths; it doesn't assume 
non-NULL part_scheme either.

* add_paths_to_append_rel(), which is called from each of the above (ie, 
generate_partitionwise_join_paths(), apply_scanjoin_target_to_paths(), 
and create_partitionwise_grouping_paths() in 
create_ordinary_grouping_paths() for creating PWJ paths, adjusting PWJ 
paths, and creating PWA paths respectively) also checks whether a rel to 
be considered has non-NULL part_scheme, but never assumes that.  And 
actually, if the rel's part_scheme is NULL, add_paths_to_append_rel() 
wouldn't be called for the rel, because of the reason described above.

Thanks for the comments, Robert!

Best regards,
Etsuro Fujita