Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jrE3J-0006jt-FA for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Jul 2020 05:15:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jrE3I-0003OV-Dv for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Jul 2020 05:15:56 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jrE3I-0003OO-26 for pgsql-hackers@lists.postgresql.org; Fri, 03 Jul 2020 05:15:56 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by magus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jrE3B-0002En-IM for pgsql-hackers@postgresql.org; Fri, 03 Jul 2020 05:15:55 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 6E9CA21C6FB7; Fri, 3 Jul 2020 08:15:48 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.129.197] (unknown [94.25.177.252]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id DC95D21C6FB4; Fri, 3 Jul 2020 08:15:47 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1593753348; bh=fZ1jbLgeDPQBQOcOosPqk0r8rNUY3xVsdhzL20SmWJk=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=qQLzMYaokL8w6GbMa/EiL13Sos1Eds3qzrhyH9JWNXQkGHF+c7I29o2J3KatgoUEl 8K8+uGt4TzUFLfCoKsso8Wl7uzwF5w7F7NyDi3WPPvE9caH1rUL0k8IgfRaNn/wOc5 7/VeUk69PCOCYenlyputVe7evNWavt6rmV1B+3YU= Subject: Re: [POC] Fast COPY FROM command for the table with foreign partitions To: Ashutosh Bapat Cc: Ashutosh Bapat , Etsuro Fujita , PostgreSQL-Dev References: <3d0909dc-3691-a576-208a-90986e55489f@postgrespro.ru> <7a3c3bdd-dc43-68a7-7cc5-ae139f1de4a7@postgrespro.ru> From: "Andrey V. Lepikhov" Organization: Postgres Professional Message-ID: <5de28d65-3c43-e460-f834-fabbfe8233a4@postgrespro.ru> Date: Fri, 3 Jul 2020 10:15:46 +0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 6/22/20 5:11 PM, Ashutosh Bapat wrote: > > wrote: > It looks like we call BeginForeignInsert and EndForeignInsert even > though actual copy is performed using BeginForeignCopy, ExecForeignCopy > and EndForeignCopy. BeginForeignInsert constructs the INSERT query which > looks unnecessary. Also some of the other PgFdwModifyState members are > initialized unnecessarily. It also gives an impression that we are using > INSERT underneath the copy. Instead a better way would be to > call BeginForeignCopy instead of BeginForeignInsert and EndForeignCopy > instead of EndForeignInsert, if we are going to use COPY protocol to > copy data to the foreign server. Corresponding postgres_fdw > implementations need to change in order to do that. > I did not answer for a long time, because of waiting for the results of the discussion on Tomas approach to bulk INSERT/UPDATE/DELETE. It seems more general. I can move the query construction into the first execution of INSERT or COPY operation. But another changes seems more invasive because BeginForeignInsert/EndForeignInsert are used in the execPartition.c module. We will need to pass copy/insert state of operation into ExecFindPartition() and ExecCleanupTupleRouting(). -- regards, Andrey Lepikhov Postgres Professional