Message-ID: <05bc8819-3e53-d884-a0d0-a17ffa5febef@postgrespro.ru>
Date: Fri, 22 Jul 2022 11:39:23 +0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: Fast COPY FROM based on batch insert
Content-Language: en-US
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Zhihong Yu <zyu@yugabyte.com>,
 Amit Langote <amitlangote09@gmail.com>, tanghy.fnst@fujitsu.com,
 Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 Pg Hackers <pgsql-hackers@postgresql.org>,
 Tomas Vondra <tomas.vondra@enterprisedb.com>, houzj.fnst@fujitsu.com
References: 
 <TYAPR01MB2990B8451B4C725242E270F7FE979@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <TYAPR01MB29900C08C2C17747612A28B5FE969@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <e3974716-e77b-9006-b1dc-dc00474d09c3@postgrespro.ru>
 <TYAPR01MB29909FDCD92F5EEE68E37767FE649@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <CALNJ-vRi4DjjkCmvmXWWtULzY6PSeEAC9vSpXVNHyVoGEQ8HVQ@mail.gmail.com>
 <20210323032322.GW4203@telsasoft.com>
 <TYAPR01MB2990124D4D859FE8C7E984C8FE649@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <20210409004956.GN6592@telsasoft.com>
 <CAPmGK15rVM+61WHZ3xS3zS6mq7293zCexVMYV4p3xoyTYo8FLg@mail.gmail.com>
 <bc489202-9855-7550-d64c-ad2d83c24867@postgrespro.ru>
 <CAPmGK15Z1kvZE0oULa+JD_CKzc4NwWEZYAfJ26VkoVYg1md63g@mail.gmail.com>
 <45ad1b94-339b-a841-8885-67ebe6fc5bc1@postgrespro.ru>
 <CAPmGK14ov-MbQtwHuSK5ESUeL1csgCVC1MAAwZxgV=NWsBE2ZQ@mail.gmail.com>
 <57733596-5fd6-07ec-d65c-67411411f10e@postgrespro.ru>
 <CAPmGK145252jR4fygMTtJcUA4fCoKPkbykir9p9XupT6N903fQ@mail.gmail.com>
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Organization: Postgres Professional
In-Reply-To: 
 <CAPmGK145252jR4fygMTtJcUA4fCoKPkbykir9p9XupT6N903fQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/05bc8819-3e53-d884-a0d0-a17ffa5febef%40postgrespro.ru>
Precedence: bulk

On 7/20/22 13:10, Etsuro Fujita wrote:
> On Tue, Jul 19, 2022 at 6:35 PM Andrey Lepikhov
> <a.lepikhov@postgrespro.ru> wrote:
>> On 18/7/2022 13:22, Etsuro Fujita wrote:
>>> I rewrote the decision logic to something much simpler and much less
>>> invasive, which reduces the patch size significantly.  Attached is an
>>> updated patch.  What do you think about that?
> 
>> maybe you forgot this code:
>> /*
>>    * If a partition's root parent isn't allowed to use it, neither is the
>>    * partition.
>> */
>> if (rootResultRelInfo->ri_usesMultiInsert)
>>          leaf_part_rri->ri_usesMultiInsert =
>>                                  ExecMultiInsertAllowed(leaf_part_rri);
> 
> I think the patch accounts for that.  Consider this bit to determine
> whether to use batching for the partition chosen by
> ExecFindPartition():
Agreed.

Analyzing multi-level heterogeneous partitioned configurations I 
realized, that single write into a partition with a trigger will flush 
buffers for all other partitions of the parent table even if the parent 
haven't any triggers.
It relates to the code:
else if (insertMethod == CIM_MULTI_CONDITIONAL &&
   !CopyMultiInsertInfoIsEmpty(&multiInsertInfo))
{
   /*
    * Flush pending inserts if this partition can't use
    * batching, so rows are visible to triggers etc.
    */
   CopyMultiInsertInfoFlush(&multiInsertInfo, resultRelInfo);
}

Why such cascade flush is really necessary, especially for BEFORE and 
INSTEAD OF triggers? AFTER Trigger should see all rows of the table, but 
if it isn't exists for parent, I think, we wouldn't obligate to 
guarantee order of COPY into two different tables.

-- 
Regards
Andrey Lepikhov
Postgres Professional