Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jlUsi-0007yc-9r for pgsql-hackers@arkaria.postgresql.org; Wed, 17 Jun 2020 10:01:20 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jlUsf-0003an-Bs for pgsql-hackers@arkaria.postgresql.org; Wed, 17 Jun 2020 10:01:17 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jlUsf-0003ag-1v for pgsql-hackers@lists.postgresql.org; Wed, 17 Jun 2020 10:01:17 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jlUsb-0000wz-JQ for pgsql-hackers@lists.postgresql.org; Wed, 17 Jun 2020 10:01:16 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 6CA1121C6D86; Wed, 17 Jun 2020 13:01:10 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.129.197] (unknown [94.25.177.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 7F48B21C6D82; Wed, 17 Jun 2020 13:01:09 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1592388070; bh=PX4tc/2102JbOoC37gVowI1rUcF8O7edC4ijuWdZLMs=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=FXC744WAzEQU/zfbyoU7F8qkDa9SItPRRrmUiib6u0pfncUdnpX045zL85snScjiu OkV4bvI0AtRQX5jBfLfzzSUPpjaFTgA8kOK3NSUXK/WvHQVW5GodJ7uqac5ofYnSFI fHNpSSftY7emXPdDRzcsEnucv97F02cR5B2wKAhU= Subject: Re: Asynchronous Append on postgres_fdw nodes. To: Kyotaro Horiguchi Cc: etsuro.fujita@gmail.com, movead.li@highgo.ca, pgsql-hackers@lists.postgresql.org References: <20200615.172934.587211238374081803.horikyota.ntt@gmail.com> <3aa386ee-99d7-551f-85cf-c13b78947d4f@postgrespro.ru> <20200616.173015.624761842205131449.horikyota.ntt@gmail.com> From: "Andrey V. Lepikhov" Organization: Postgres Professional Message-ID: <581520e2-1475-c614-b879-98b350220508@postgrespro.ru> Date: Wed, 17 Jun 2020 15:01:08 +0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <20200616.173015.624761842205131449.horikyota.ntt@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 6/16/20 1:30 PM, Kyotaro Horiguchi wrote: > They return 25056 rows, which is far more than 9741 rows. So remote > join won. > > Of course the number of returning rows is not the only factor of the > cost change but is the most significant factor in this case. > Thanks for the attention. I see one slight flaw of this approach to asynchronous append: AsyncAppend works only for ForeignScan subplans. if we have PartialAggregate, Join or another more complicated subplan, we can't use asynchronous machinery. It may lead to a situation than small difference in a filter constant can cause a big difference in execution time. I imagine an Append node, that can switch current subplan from time to time and all ForeignScan nodes of the overall plan are added to one queue. The scan buffer can be larger than a cursor fetch size and each IterateForeignScan() call can induce asynchronous scan of another ForeignScan node if buffer is not full. But these are only thoughts, not an proposal. I have no questions to your patch right now. -- Andrey Lepikhov Postgres Professional https://postgrespro.com