Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lq8sE-0007Lm-0q for pgsql-hackers@arkaria.postgresql.org; Mon, 07 Jun 2021 06:36:34 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lq8s9-0000nG-Th for pgsql-hackers@arkaria.postgresql.org; Mon, 07 Jun 2021 06:36:29 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lq8s9-0000n6-Hg for pgsql-hackers@lists.postgresql.org; Mon, 07 Jun 2021 06:36:29 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lq8s3-00032F-9F for pgsql-hackers@postgresql.org; Mon, 07 Jun 2021 06:36:28 +0000 Received: from Andreys-MacBook-Pro.local (unknown [176.59.207.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id EAD5E21C81EF; Mon, 7 Jun 2021 09:36:19 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1623047780; bh=4yU/JtIG3qm5zMplgCmm1d0LRhUtwTFQ2+czxRfAm4I=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=ABHp3DwQ7YmVV1Izq4Bp6LePqAhislkvjj8hZyJ3gBB0X7ZjKRBeILicvUc0deB31 I7IuHC2j5s9q1GCD9mhbSWgvfnJXD8eLt1pbGxE+6YzwKDQIg11AhWdacfTBO/PC4g 22q7l7Pspr8ghwyyBAOz6WlhVVFDLLDN6NN+m+xY= Subject: Re: Fast COPY FROM based on batch insert To: "tsunakawa.takay@fujitsu.com" Cc: Etsuro Fujita , Justin Pryzby , Zhihong Yu , Amit Langote , "tanghy.fnst@fujitsu.com" , Ashutosh Bapat , Pg Hackers , Tomas Vondra , "houzj.fnst@fujitsu.com" References: <20210304203949.GQ29832@telsasoft.com> <20210323032322.GW4203@telsasoft.com> <20210409004956.GN6592@telsasoft.com> From: Andrey Lepikhov Organization: Postgres Professional Message-ID: Date: Mon, 7 Jun 2021 11:36:18 +0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 4/6/21 13:45, tsunakawa.takay@fujitsu.com wrote: > From: Andrey Lepikhov >> We still have slow 'COPY FROM' operation for foreign tables in current master. >> Now we have a foreign batch insert operation And I tried to rewrite the patch [1] >> with this machinery. > > I haven't looked at the patch, but nice performance. > > However, I see the following problems. What do you think about them? I agree with your fears. Think about this patch as an intermediate step on the way to fast COPY FROM. This patch contains all logic of the previous patch, except of transport machinery (bulk insertion api). It may be simpler to understand advantages of proposed 'COPY' FDW API having committed 'COPY FROM ...' feature based on the bulk insert FDW API. -- regards, Andrey Lepikhov Postgres Professional