From: Andrey V. Lepikhov <a.lepikhov@postgrespro.ru>
To: Ashutosh Bapat <ashutosh.bapat@2ndquadrant.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: PostgreSQL-Dev <pgsql-hackers@postgresql.org>
Subject: Re: [POC] Fast COPY FROM command for the table with foreign partitions
Date: Fri, 3 Jul 2020 10:15:46 +0500
Message-ID: <5de28d65-3c43-e460-f834-fabbfe8233a4@postgrespro.ru> (raw)
In-Reply-To: <CAG-ACPUZ1FRwkRfOnBZN0PYvXSZrBRgaExyRT4QHaN20rf7A8Q@mail.gmail.com>
References: <3d0909dc-3691-a576-208a-90986e55489f@postgrespro.ru>
<CAPmGK15mAiaU85OcvEDE=7fHA3PvA7ChP82b7bEE57Yw20rF4A@mail.gmail.com>
<7a3c3bdd-dc43-68a7-7cc5-ae139f1de4a7@postgrespro.ru>
<CAExHW5uAtyAVL-iuu1Hsd0fycqS5UHoHCLfauYDLQwRucwC9Og@mail.gmail.com>
<bd29904a-4ad3-3619-eb0c-ff04f524f7a1@postgrespro.ru>
<CAG-ACPUZ1FRwkRfOnBZN0PYvXSZrBRgaExyRT4QHaN20rf7A8Q@mail.gmail.com>
On 6/22/20 5:11 PM, Ashutosh Bapat wrote:
> <a.lepikhov@postgrespro.ru <mailto:a.lepikhov@postgrespro.ru>> wrote:
> It looks like we call BeginForeignInsert and EndForeignInsert even
> though actual copy is performed using BeginForeignCopy, ExecForeignCopy
> and EndForeignCopy. BeginForeignInsert constructs the INSERT query which
> looks unnecessary. Also some of the other PgFdwModifyState members are
> initialized unnecessarily. It also gives an impression that we are using
> INSERT underneath the copy. Instead a better way would be to
> call BeginForeignCopy instead of BeginForeignInsert and EndForeignCopy
> instead of EndForeignInsert, if we are going to use COPY protocol to
> copy data to the foreign server. Corresponding postgres_fdw
> implementations need to change in order to do that.
>
I did not answer for a long time, because of waiting for the results of
the discussion on Tomas approach to bulk INSERT/UPDATE/DELETE. It seems
more general.
I can move the query construction into the first execution of INSERT or
COPY operation. But another changes seems more invasive because
BeginForeignInsert/EndForeignInsert are used in the execPartition.c
module. We will need to pass copy/insert state of operation into
ExecFindPartition() and ExecCleanupTupleRouting().
--
regards,
Andrey Lepikhov
Postgres Professional
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.lepikhov@postgrespro.ru, ashutosh.bapat@2ndquadrant.com, ashutosh.bapat.oss@gmail.com, etsuro.fujita@gmail.com
Subject: Re: [POC] Fast COPY FROM command for the table with foreign partitions
In-Reply-To: <5de28d65-3c43-e460-f834-fabbfe8233a4@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox