Message-ID: <5B3DFEFB.1080007@lab.ntt.co.jp>
Date: Thu, 05 Jul 2018 20:20:27 +0900
From: Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:11.0) Gecko/20120327 Thunderbird/11.0.1
MIME-Version: 1.0
Subject: Re: Expression errors with "FOR UPDATE" and postgres_fdw with
 partition
 wise join enabled.
References: 
 <CAKcux6ktu-8tefLWtQuuZBYFaZA83vUzuRd7c1YHC-yEWyYFpg@mail.gmail.com>
 <5AF2E09F.9060208@lab.ntt.co.jp>
 <CAFjFpRcFucgvrzEFNi10g00-va640poSmJDmBrZNfFarB6eP-Q@mail.gmail.com>
 <5AF440C2.3040805@lab.ntt.co.jp>
 <CAFjFpRczEuG9b0_j_qve2JAi-HHqS9_m1_yK5QVQa5=oDr5mJA@mail.gmail.com>
 <5AF59415.10309@lab.ntt.co.jp>
 <CAFjFpRfkafBFJSEEkGwzLf=-y9Q8TUEGfpR=+K7-zCOw80R10w@mail.gmail.com>
 <CAFjFpRc8ZoDm0+zhx+MckwGyEqkOzWcpVqbvjaxwdGarZSNrmA@mail.gmail.com>
 <5AFC0865.8050802@lab.ntt.co.jp>
 <CAFjFpRfGL9w-4C5N-ioaG3m+iheC1ey7_RMZ4vJijGn=wWJyxw@mail.gmail.com>
 <5AFD6580.5090308@lab.ntt.co.jp> <5AFE81CC.8000508@lab.ntt.co.jp>
 <5B17C5EC.1090602@lab.ntt.co.jp>
 <CAFjFpRffA-BfL0RXcUVLLbcOBF1ie7oObTB-RE2q_o45X1HTYg@mail.gmail.com>
 <5B1E5071.1000704@lab.ntt.co.jp> <5B23A964.9020800@lab.ntt.co.jp>
 <5B28FB19.3090809@lab.ntt.co.jp> <5B39F47B.10707@lab.ntt.co.jp>
 <5B3C9C33.6030203@lab.ntt.co.jp>
 <CAFjFpRfVwaxfrRi8u5LmBX-zMhyzwYdGcj2VzpoxKYWLm9Y91A@mail.gmail.com>
 <5B3DFDDE.2020803@lab.ntt.co.jp>
In-Reply-To: <5B3DFDDE.2020803@lab.ntt.co.jp>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
To: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
        Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Precedence: bulk

(2018/07/05 20:15), Etsuro Fujita wrote:
> (2018/07/04 19:11), Ashutosh Bapat wrote:
>> On Wed, Jul 4, 2018 at 3:36 PM, Etsuro Fujita
>> <fujita.etsuro@lab.ntt.co.jp> wrote:
>>>
>>> I don't produce a test case where the plan is an Append with Gather
>>> subplans, but I'm not sure that it's a good thing to allow us to
>>> consider
>>> such a plan because in that plan, each Gather would try to grab its
>>> own pool
>>> of workers. Am I missing something?
>>
>> If the overall join can not use parallelism, having individual child
>> joins use parallel query might turn out to be efficient even if done
>> one child at a time. Parallel append drastically reduces the cases
>> where something like could be useful, but I don't think we can
>> theoretically eliminate the need for such a plan.
>
> In the case where scanjoin_target_parallel_safe=false, we actually will
> consider such parallel paths using the existing partial paths for the
> parent appendrel in the code path shown in a previous email (note: we
> would already have done generate_partitionwise_join_paths or
> set_append_rel_pathlist for the parent appendrel in query_planner.) For
> such a parallel path, we might need to do a projection in the Gather
> node for generating the SRF-free scan/join target, but I think that
> would be still efficient. So, I'm not sure that we really need to create
> child Gathers and generate an Append with those Gathers, as you mentioned.
>
> Another thing I noticed is: actually, we don't produce an Append with
> child Gathers in apply_scanjoin_target_to_paths, which I thought we
> would do that in the case of scanjoin_target_parallel_safe=false, but I
> noticed I was wrong. Sorry for that. The reason is because in that case,
> even if we create new partial Append paths with child Gathers, we don't
> run generate_gatehr_paths for the newly created partial paths at the end
> of that function shown below, since the parent's consider_parallel flag
> is set to false in that case:
>
> /*
> * Consider generating Gather or Gather Merge paths. We must only do this
> * if the relation is parallel safe, and we don't do it for child rels to
> * avoid creating multiple Gather nodes within the same plan. We must do
> * this after all paths have been generated and before set_cheapest, since
> * one of the generated paths may turn out to be the cheapest one.
> */
> if (rel->consider_parallel && !IS_OTHER_REL(rel))
> generate_gather_paths(root, rel, false);

One thing to add is: ISTM we need some cleanup for 
apply_scanjoin_target_to_paths.

Best regards,
Etsuro Fujita