Subject: Re: POC: postgres_fdw insert batching
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers@postgresql.org
References: <20200628151002.7x5laxwpgvkyiu3q@development>
From: "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>
Organization: Postgres Professional
Message-ID: <0300d7df-854b-6f72-1e93-dfbf98f3fc8a@postgrespro.ru>
Date: Fri, 10 Jul 2020 09:28:44 +0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.8.0
MIME-Version: 1.0
In-Reply-To: <20200628151002.7x5laxwpgvkyiu3q@development>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 6/28/20 8:10 PM, Tomas Vondra wrote:
> Now, the primary reason why the performance degrades like this is that
> while FDW has batching for SELECT queries (i.e. we read larger chunks of
> data from the cursors), we don't have that for INSERTs (or other DML).
> Every time you insert a row, it has to go all the way down into the
> partition synchronously.

You added new fields into the PgFdwModifyState struct. Why you didn't 
reused ResultRelInfo::ri_CopyMultiInsertBuffer field and 
CopyMultiInsertBuffer machinery as storage for incoming tuples?

-- 
regards,
Andrey Lepikhov
Postgres Professional