pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Ian Barwick <ian.barwick@enterprisedb.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: Zhihong Yu <zyu@yugabyte.com>
Cc: Amit Langote <amitlangote09@gmail.com>
Cc: tanghy.fnst@fujitsu.com, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: houzj.fnst@fujitsu.com
Subject: Re: Fast COPY FROM based on batch insert
Date: Fri, 8 Jul 2022 18:09:35 +0300
Message-ID: <b3a60e56-4d33-c2cb-3708-f234392be3a4@postgrespro.ru> (raw)
In-Reply-To: <17be446f-65a5-b49e-e57c-5629431e26b1@enterprisedb.com>
References: <TYAPR01MB2990B8451B4C725242E270F7FE979@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<20210304203949.GQ29832@telsasoft.com>
	<TYAPR01MB29900C08C2C17747612A28B5FE969@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<e3974716-e77b-9006-b1dc-dc00474d09c3@postgrespro.ru>
	<TYAPR01MB29909FDCD92F5EEE68E37767FE649@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<CALNJ-vRi4DjjkCmvmXWWtULzY6PSeEAC9vSpXVNHyVoGEQ8HVQ@mail.gmail.com>
	<20210323032322.GW4203@telsasoft.com>
	<TYAPR01MB2990124D4D859FE8C7E984C8FE649@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<20210409004956.GN6592@telsasoft.com>
	<CAPmGK15rVM+61WHZ3xS3zS6mq7293zCexVMYV4p3xoyTYo8FLg@mail.gmail.com>
	<bc489202-9855-7550-d64c-ad2d83c24867@postgrespro.ru>
	<CAPmGK15Z1kvZE0oULa+JD_CKzc4NwWEZYAfJ26VkoVYg1md63g@mail.gmail.com>
	<45ad1b94-339b-a841-8885-67ebe6fc5bc1@postgrespro.ru>
	<065fde84-f688-c411-28eb-a3b9084b38b7@enterprisedb.com>
	<811b36f0-8888-7036-f4c5-0bfbdfad0bbf@postgrespro.ru>
	<17be446f-65a5-b49e-e57c-5629431e26b1@enterprisedb.com>

On 8/7/2022 05:12, Ian Barwick wrote:
>      ERROR:  bind message supplies 0 parameters, but prepared statement 
> "pgsql_fdw_prep_178" requires 6
>      CONTEXT:  remote SQL command: INSERT INTO public.foo_part_1(t, v1, 
> v2, v3, v4, v5) VALUES ($1, $2, $3, $4, $5, $6)
>      COPY foo, line 88160
Thanks, I got it. MultiInsertBuffer are created on the first non-zero 
flush of tuples into the partition and isn't deleted from the buffers 
list until the end of COPY. And on a subsequent flush in the case of 
empty buffer we catch the error.
Your fix is correct, but I want to propose slightly different change 
(see in attachment).

-- 
regards,
Andrey Lepikhov
Postgres Professional

Attachments:


flush_nonzero_buffer.txt (text/plain, 1.2K) download
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 245a260982..203289f7f2 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -329,7 +329,8 @@ CopyMultiInsertBufferFlush(CopyMultiInsertInfo *miinfo,
                           resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize != NULL);
 
                /* Flush into foreign table or partition */
-               do {
+               while(sent < nused)
+               {
                        int size = (resultRelInfo->ri_BatchSize < nused - sent) ?
                                                resultRelInfo->ri_BatchSize : (nused - sent);
                        int inserted = size;
@@ -340,7 +341,7 @@ CopyMultiInsertBufferFlush(CopyMultiInsertInfo *miinfo,
                                                                                                                                 NULL,
                                                                                                                                 &inserted);
                        sent += size;
-               } while (sent < nused);
+               }
        }
        else
        {

view thread (39+ messages)  latest in thread

Message-ID: <b3a60e56-4d33-c2cb-3708-f234392be3a4@postgrespro.ru>
Permalink:  ../b3a60e56-4d33-c2cb-3708-f234392be3a4@postgrespro.ru/
Also on:    postgresql.org/message-id/b3a60e56-4d33-c2cb-3708-f234392be3a4@postgrespro.ru

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: a.lepikhov@postgrespro.ru, ian.barwick@enterprisedb.com, pryzby@telsasoft.com, zyu@yugabyte.com, amitlangote09@gmail.com, ashutosh.bapat.oss@gmail.com, tomas.vondra@enterprisedb.com, etsuro.fujita@gmail.com, houzj.fnst@fujitsu.com
  Subject: Re: Fast COPY FROM based on batch insert
  In-Reply-To: <b3a60e56-4d33-c2cb-3708-f234392be3a4@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox