Subject: Re: New Table Access Methods for Multi and Single Inserts
To: Jeff Davis <pgsql@j-davis.com>,
 Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Andres Freund <andres@anarazel.de>, Paul Guo <guopa@vmware.com>
References: 
 <CALj2ACVi9eTRYR=gdca5wxtj3Kk_9q9qVccxsS1hngTGOCjPwQ@mail.gmail.com>
 <20201217050522.GU30237@telsasoft.com>
 <CALj2ACVgT1iocd5nQ+rEmqt3xcCONkR037qbc8PiojdR39Ag=w@mail.gmail.com>
 <20201217204442.GX30237@telsasoft.com>
 <CALj2ACW3BC5kgdffZ2LD_CT2wQoXVc29kGB74SVWnGZ=UFqcAQ@mail.gmail.com>
 <20201218175439.GA30237@telsasoft.com> <20201221074725.GF30237@telsasoft.com>
 <CALj2ACWMnZZCu=G0PJkEeYYicKeuJ-X=SU19i6vQ1+=uXz8u0Q@mail.gmail.com>
 <20201225023958.GW30237@telsasoft.com>
 <CALj2ACVDtYYRYD2SC+X2ALOUkhnUcgC7RLxiEYVWW2HxxrfRww@mail.gmail.com>
 <96eaa813-4ad6-e80a-04a4-cc8082d356dd@swarm64.com>
 <CALj2ACVsiAZMsP8p5MPg6SSEtoMFFaiAa6j2AFtEQJDhfbgs3Q@mail.gmail.com>
 <508af801-6356-d36b-1867-011ac6df8f55@swarm64.com>
 <CALj2ACUmL3+xLFtVbdcNpo_=ubdi=_nsp6MNq__xWwL=NGkdgA@mail.gmail.com>
 <c54a69a8b60bae42499b86ff771ecbcdad29326d.camel@j-davis.com>
From: Luc Vlaming <luc@swarm64.com>
Message-ID: <2281baee-081c-7360-cb51-4f16e1e91d8c@swarm64.com>
Date: Mon, 18 Jan 2021 08:58:08 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.5.0
In-Reply-To: <c54a69a8b60bae42499b86ff771ecbcdad29326d.camel@j-davis.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [IPv6:2a02:8109:a080:13f4:56ee:75ff:fe80:b7f2]
 (2a02:8109:a080:13f4:56ee:75ff:fe80:b7f2) by
 FRXPR01CA0029.DEUPRD01.PROD.OUTLOOK.DE (2a01:4180:c010:14::39) with Microsoft
 SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.6 via Frontend
 Transport; Mon, 18 Jan 2021 07:58:09 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a167624-001f-4a34-0e55-08d8bb86cc12
X-MS-TrafficTypeDiagnostic: FRXPR01MB0071:
X-Microsoft-Antispam-PRVS: 
	<FRXPR01MB007129B50982C7C57BE740A8B7A40@FRXPR01MB0071.DEUPRD01.PROD.OUTLOOK.DE>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	oI+Xjzf7Mr9/hMzJGCyOQ6rsuPxZAaOzzP8Hj37uCZ8OnQBkukSFjqfSuhh28x5QOVyISpDQcb5uDQ/DUyJvtb8UdGCPDajWL23pL+YcOiDNWHpza8BeKocuGXBcZHMzQfr4XDarM2iPagzl/DqtarH3P0PiJxV5ixTJmYqs7SloMfWL7ZuclCBkDOFHE0sL3wEftSKswbbHZ+gk+kVXlUSLSYtaKqyrfpz3m+HvExgzkzeZotMJSGZ84DBMIlldv+IUIsacXBdkd8zXpCvxYPaDRG0+cvYE2a38lzMLSPfU/1NYnbocT1DmdNUFQI7W7nzCB3hCK6g/wnzvdSN7/Dtj4wyRms39FmWsOLoYPyM/6SDiooMSdPHKPIx7575mr4icH06feST6hRnhTvFWw84HUbdw+1deEda+iSDjyKvFPlqp/rUlIy1Xs6e/5uQA
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:FRXPR01MB0456.DEUPRD01.PROD.OUTLOOK.DE;PTR:;CAT:NONE;SFS:(39830400003)(346002)(396003)(136003)(376002)(366004)(8676002)(316002)(66476007)(186003)(54906003)(66556008)(52116002)(110136005)(2906002)(478600001)(16526019)(5660300002)(8936002)(4326008)(86362001)(53546011)(66946007)(31696002)(36756003)(31686004)(2616005)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: 
	=?utf-8?B?d244VGF5NXpTRzE2RVU3anV2S1lmQXdpcDl5U0p0aVdjN0lQSDYxRTNMUERt?=
 =?utf-8?B?ZU9FNm0vVXRmOEZpeTMxaFc4Ukl5YmdhcVU2TlpkeTB4ZzJQVDdjS1F6Z3No?=
 =?utf-8?B?V29wRjg2cFFtK2JaVXNOZVFHdzZrbXpNUnQwS25OL0paQ1B3R2Fodks4emI4?=
 =?utf-8?B?VklST0tKZDkyQlRKUmdsWHUzNHFLellqWkZMejdsUXNQZnZPVmJyMkpuc1Ni?=
 =?utf-8?B?U2w3SGFYQ1MySHVUS0ZRc25ZYW9LTkRMT3U0dXVia29zTVlPK2x6SVhVZmJL?=
 =?utf-8?B?N2pFdThvN3ByUzA3VTRhSDZmS2RXaE52SFhkVitiSHFJS2cyV245SGk2UjZL?=
 =?utf-8?B?RzNNYUVoSzg1ODRXNVNpVnJXSktOdjUxMHBQNHZxb2xNdkZoeVBoNmluYUw3?=
 =?utf-8?B?bVdiOVl2N0Qzb2Q1d29IWmYzeWpjZ1pkckJpOW55cEx5TnFndmVhbGpkMVBW?=
 =?utf-8?B?MGI3S0pvbjVDTHlTaVlGZWFxYWtjYVo2SDdvWVhxbzFYOWZTYm1mWmNmaHdv?=
 =?utf-8?B?bUcvVmtUWWxUVnlmamRodm9vRm5NaDl1aUNHVy81bkFJU2l0dW1TNEZocWlB?=
 =?utf-8?B?L1o2K3d0Q0pkS1lrbERTL0VwR0RKNnQ0SW5obWY5S0VsdzU3Tk5vTmtUdWl6?=
 =?utf-8?B?VWl4RUxLYlc3aUs5cmIwYVR1cm9KV3g3dElrM1BNNWRXQU8rSXdsM1k1MUZQ?=
 =?utf-8?B?a3RKTkpXdU5Fdy9weXV6SFFZdFJ4ckZ0Wlpodm94UDZvRzIyb3ZRT3ZvY0pZ?=
 =?utf-8?B?eDhnYUMwaUZzeExXcTF4UFlYR0QyTmM3QTB1Q1VLTUovZzlVc2tzMm1mdkh4?=
 =?utf-8?B?MUJacTJyb3BaMGN4dU1YbW1iQ1grbUFmS0gzQXdJY0J6R0p0dEtOK2hnQ1Vj?=
 =?utf-8?B?N1VsTGhvMDk2UGc4MWtQejg1L3phSFY0Tk5uWFhOOFJ5WEVXN3pZd2lYVFpH?=
 =?utf-8?B?R3BYQkhHUXBweVhIWlFPU1grb0JURHc4R1ZhUGRWaStVdGJPV0k5NWdEMFEx?=
 =?utf-8?B?ZEJNWjdHQ2lpNDh3Z09ib3R6ZURRUCs1eXNCZWg1NGNrMHBzamFSRFUyL3Bl?=
 =?utf-8?B?OXpDUGxxbDdpcVJKeGFRVDdHV1AvL2J4bDA0VGlKKys1Z3MvelI4Nkh0U1pv?=
 =?utf-8?B?UlZYZ3VHa213bHR2ZmhMY3N1RkNxbmtWZUsrOU83dEE3dDZtM1dKWHhjN1Np?=
 =?utf-8?B?UmJDd3pYbld3QVVRS3liOGxkS1ByWGE4TjdQN0FXbVQ0SnNKQmhFdis2dTZk?=
 =?utf-8?B?eFVrL2lWNTFCbGdPUnBvLzg1TUJRZ1c3cGFpYVk4M2x2UTlwTFo5bXl6NXla?=
 =?utf-8?B?TnA4d0V0azk4MDVnZlZDZ1VnMVJCcFRoczhpTmdqVjB6Nm80VDljZUp6bng3?=
 =?utf-8?B?VzVyblA1Q0I2Mjl0N2hIbEx0dDBtRldLTlpGUmtYQnR6MHhUZzVzRE96Ukcr?=
 =?utf-8?Q?qq/6evw/?=
X-OriginatorOrg: swarm64.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 
 1a167624-001f-4a34-0e55-08d8bb86cc12
X-MS-Exchange-CrossTenant-AuthSource: FRXPR01MB0456.DEUPRD01.PROD.OUTLOOK.DE
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 07:58:09.7791
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 6a3a733d-0e05-4774-8f9e-4fc6a12face6
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 
 8JuhAl+T6GCvst0REEV9P0G097TyzIHACLQboKU22fK+4XBOdQqRjYA9bUj2RZxX
X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRXPR01MB0071
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Precedence: bulk

On 17-01-2021 00:04, Jeff Davis wrote:
> 
>> If we agree on removing heap_multi_insert_v2 API and embed that logic
>> inside heap_insert_v2, then we can do this - pass the required
>> information and the functions ExecInsertIndexTuples and
>> ExecARInsertTriggers as callbacks so that, whether or not
>> heap_insert_v2 choses single or multi inserts, it can callback these
>> functions with the required information passed after the flush. We
>> can
>> add the callback and required information into TableInsertState. But,
>> I'm not quite sure, we would make ExecInsertIndexTuples and
>> ExecARInsertTriggers.
> 
> How should the API interact with INSERT INTO ... SELECT? Right now it
> doesn't appear to be integrated at all, but that seems like a fairly
> important path for bulk inserts.
> 
> Regards,
> 	Jeff Davis
> 
> 

Hi,

You mean how it could because of that the table modification API uses 
the table_tuple_insert_speculative ? Just wondering if you think if it 
generally cannot work or would like to see that path / more paths 
integrated in to the patch.

Kind regards,
Luc