pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Amit Langote <amitlangote09@gmail.com>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
To: Tsunakawa, Takayuki <tsunakawa.takay@fujitsu.com>
Subject: Re: POC: postgres_fdw insert batching
Date: Thu, 18 Feb 2021 00:38:08 +0100
Message-ID: <f44047f8-3bc9-51e1-61d8-7aaae56aee5a@enterprisedb.com> (raw)
In-Reply-To: <6929d485-2d2a-da46-3681-4a400a3d794f@enterprisedb.com>
References: <20200628151002.7x5laxwpgvkyiu3q@development>
	<ebbd67c6-054a-f5eb-39a9-95e088f1c985@enterprisedb.com>
	<TYAPR01MB29903DF617C2B868C2477738FEA10@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<0fc809f6-7c3c-6d06-7ea3-f10765dfdf08@enterprisedb.com>
	<CA+HiwqHTnfTG52XxNZriFTr+1r1VyrLm_u-xU5cUuXW-XuLUcA@mail.gmail.com>
	<067ddf09-225c-e3c5-ea47-4dd39b34ebb0@enterprisedb.com>
	<CA+HiwqHrGA0HB5cjBL_0=Gk-u-dH=K8beAPibhOCFepU0kqndw@mail.gmail.com>
	<TYAPR01MB2990E3D10A122358419482B2FEBD0@TYAPR01MB2990.jpnprd01.prod.outlook.com>
	<CA+HiwqGjMVLEn8JoNT+XCeXA9SFECVvTEMawbzJ-RN_M+fxMTw@mail.gmail.com>
	<7ffcee1f-d55e-cbdf-5f3f-98f1c30537ef@enterprisedb.com>
	<CA+HiwqF4cB5AbJRW2RwqQfeOXwg0KjLB5t3Gh9JCiR7J-N8HwA@mail.gmail.com>
	<803d5507-299d-f109-7b71-b6bbbb17a053@enterprisedb.com>
	<CA+HiwqEPWAqheRFpyfQf2uO4cyhDEvhEwsc04_7GAxDuRGdBQg@mail.gmail.com>
	<CA+HiwqGyr_WKQ+-eR+hEHUwm4Frm==spQrdZtZU1S8WKT3YBaw@mail.gmail.com>
	<6929d485-2d2a-da46-3681-4a400a3d794f@enterprisedb.com>

On 2/17/21 8:36 PM, Tomas Vondra wrote:
>
> ...
> 
> Thanks. The patch seems reasonable, but it's a bit too large for a fix,
> so I'll go ahead and push one of the previous fixes restricting batching
> to plain INSERT commands. But this seems useful, so I suggest adding it
> to the next commitfest.

I've pushed the v4 fix, adding the CMD_INSERT to execPartition.

I think we may need to revise the relationship between FDW and places
that (may) call GetForeignModifyBatchSize. Currently, these places need
to be quite well synchronized - in a way, the issue was (partially) due
to postgres_fdw and core not agreeing on the details.

In particular, create_foreign_modify sets batch_size for CMD_INSERT and
leaves it 0 otherwise. So GetForeignModifyBatchSize() returned 0 later,
triggering an assert.

It's probably better to require GetForeignModifyBatchSize() to always
return a valid batch size (>= 1). If batching is not supported, just
return 1.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





view thread (115+ messages)  latest in thread

Message-ID: <f44047f8-3bc9-51e1-61d8-7aaae56aee5a@enterprisedb.com>
Permalink:  ../f44047f8-3bc9-51e1-61d8-7aaae56aee5a@enterprisedb.com/
Also on:    postgresql.org/message-id/f44047f8-3bc9-51e1-61d8-7aaae56aee5a@enterprisedb.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@enterprisedb.com, amitlangote09@gmail.com, tsunakawa.takay@fujitsu.com
  Subject: Re: POC: postgres_fdw insert batching
  In-Reply-To: <f44047f8-3bc9-51e1-61d8-7aaae56aee5a@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox