Subject: Re: POC: postgres_fdw insert batching
To: Zhihong Yu <zyu@yugabyte.com>
Cc: Amit Langote <amitlangote09@gmail.com>,
 "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>,
 "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>,
 Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
References: <20200628151002.7x5laxwpgvkyiu3q@development>
 <OSBPR01MB29820BAA17AD3A79A49935EFFEA40@OSBPR01MB2982.jpnprd01.prod.outlook.com>
 <16fcfd4c-2d5e-67f7-fa69-9b5d119bac52@enterprisedb.com>
 <TYAPR01MB299026DB0E91E513664C162FFEA30@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <ba58f299-d50b-f347-4a0f-778cb6ac5588@enterprisedb.com>
 <TYAPR01MB2990A053D902EDF6244090BDFEA30@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <CA+HiwqH-7+L0B7iNWcAe8auWch=-EnbRkLYMG-27K4rUKsWKOQ@mail.gmail.com>
 <TYAPR01MB29903CBD68C9DA42B67D9EB6FEA30@TYAPR01MB2990.jpnprd01.prod.outlook.com>
 <CA+HiwqGiwxSc8kc0PgPef5DgnP7NTRHVyAZJG3zxY4XBivbfLQ@mail.gmail.com>
 <9d94c863-ce7c-ea7f-6b84-834c98929e2f@enterprisedb.com>
 <CA+HiwqHTeQKqRpCA3TGZSCYTVOcUoUdXeSrRfjRRcmT1CfM1oA@mail.gmail.com>
 <7e70a11e-cf5d-2e5f-a802-274c5e0e236b@enterprisedb.com>
 <eb81a728-4ac3-823e-0017-54a9491f5600@enterprisedb.com>
 <CALNJ-vSt7Fn=oj4NxAyCDQQF-1yxmP=LYyQwciApg4L+QdYJQw@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
Message-ID: <3ca9496f-0827-bf21-b709-593de63440a5@enterprisedb.com>
Date: Thu, 21 Jan 2021 01:56:25 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.6.0
MIME-Version: 1.0
In-Reply-To: 
 <CALNJ-vSt7Fn=oj4NxAyCDQQF-1yxmP=LYyQwciApg4L+QdYJQw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 1/21/21 1:17 AM, Zhihong Yu wrote:
> Hi,
> The assignment to resultRelInfo is done when junk_filter_needed is true:
> 
>          if (junk_filter_needed)
>          {
>              resultRelInfo = mtstate->resultRelInfo;
> 
> Should the code for determining batch size access mtstate->resultRelInfo 
> directly ?
> 

IMO the issue is that code iterates over all plans and moves to the next 
for each one:

     resultRelInfo++;

so it ends up pointing past the last element, hence the failures. So 
yeah, either the code needs to move before the loop (per my patch), or 
we need to access mtstate->resultRelInfo directly.

I'm pretty amazed this did not crash during any of the many regression 
runs I did recently.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company