Re: Speed up COPY FROM text/CSV parsing using SIMD

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Manni Wood <[email protected]>
To: KAZAR Ayoub <[email protected]>
Cc: Nazir Bilal Yavuz <[email protected]>
Cc: Mark Wong <[email protected]>
Cc: Neil Conway <[email protected]>
Cc: Nathan Bossart <[email protected]>
Cc: Andrew Dunstan <[email protected]>
Cc: Shinya Kato <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
Date: Wed, 4 Feb 2026 09:07:07 -0600
Message-ID: <CAKWEB6r1a3yacHx8bAM3qfpUps4=rm+uUC7JxBfx3P5J0r_SdA@mail.gmail.com> (raw)
In-Reply-To: <CA+K2RunttnPQShNKcz5xH_4UaTM=Lomxv1EJ8RJyZsKsxmvaWA@mail.gmail.com>
References: <aTx-LDyiHV-7wfOP@ltdrgnflg2>
	<CAKWEB6r=axZsG-s7zyWURZ-s9-s1dTV9ohkZXO0ynfLEU5ha3Q@mail.gmail.com>
	<CAN55FZ2DE2XSrFUhsOqbpBo+BtzTwsJWOD0MffvdGnHtbsPRuw@mail.gmail.com>
	<CA+K2RumOaH-daBGN6uTo6+_0XSg7HQ10Na8OzScCV5j6eKkFgA@mail.gmail.com>
	<CAN55FZ3fWSk0h09Yfbb2eO4COfQDSL7Ofk7xF3q_Wg4ags3kPw@mail.gmail.com>
	<CAKWEB6qQmPejLwndzMcVqhO+u9vgUf44kYAKg4QtyG-=KH=YGg@mail.gmail.com>
	<CAKWEB6pW6xJkVwUg8zByFg7U_ZPhTJxQnLyBOMFSc7ttWbu3tQ@mail.gmail.com>
	<CAN55FZ1g1sAtZ58mk9GJFqVEi4cBFoCdMwC3CbJLq1wvtXu61A@mail.gmail.com>
	<aWaZIWAU_wIHg7FO@o>
	<CAKWEB6o2WgXAJyWU-wdMOrv=VD26Hs3NutANhEcJbKbKWpEXEg@mail.gmail.com>
	<aX0rLfLf3aFO4cl-@o>
	<CA+K2RunBvVQGPvxGLBf60YV_6b2goeG3Y9+CS6XVaxaJyWCNGw@mail.gmail.com>
	<CAN55FZ0EiPgUTJmGQXm9gP_VnogdQMkg2yhZSvwoSFJ1uVhFFg@mail.gmail.com>
	<CAN55FZ2PD5ZuvcrJPzZhZ==VDYPVwrPKNqbeDsiA4NKncrzn1g@mail.gmail.com>
	<CA+K2Ru=BNL=CEJpmfTsyqfCwzUTErU3+qsQ91aQh=njK6CsKvQ@mail.gmail.com>
	<CAKWEB6pdser1_ewZHyb+bK7QimFmAPiq8V+1kOsXSqRj0KSeHQ@mail.gmail.com>
	<CA+K2RunttnPQShNKcz5xH_4UaTM=Lomxv1EJ8RJyZsKsxmvaWA@mail.gmail.com>

On Wed, Feb 4, 2026 at 8:29 AM KAZAR Ayoub <[email protected]> wrote:

> Hello,
>
> On Wed, Feb 4, 2026, 6:38 AM Manni Wood <[email protected]>
> wrote:
>
>> The 0001-COPY-from-SIMD-v3-with-line_buf-periodic-refill.patch seems
>> nice! On My x86 PC, it had the usual performance improvment of earlier
>> patches, but the regression seemed more similar for both text and csv
>> inputs. Unfortunately, the regression is about 2.5%, but maybe that is an
>> acceptable worst-case for an improvement of 22% for text inputs and 33% for
>> CSV inputs?
>>
>> The 0001-COPY-from-SIMD-v3-with-line_buf-periodic-refill.patch looks even
>> better on my Raspberry Pi's arm processor: not only do we see a 22%
>> improvement for text and an almost 34% improvement for CSV, even the
>> worst-case scenarios show an almost 4% improvement for text and an 11.7%
>> improvement for CSV.
>>
>> By comparison,
>> the v5.1-0001-Simple-heuristic-for-SIMD-COPY-FROM.patch.patch's worst-case
>> performance is poorer on both architectures.
>>
>> I'd be curious to know if anyone else can reproduces these
>> numbers. 0001-COPY-from-SIMD-v3-with-line_buf-periodic-refill.patch seems
>> like a real winner.
>>
> Thanks for the benchmark Manni, i suppose this is the same threshold as
> patch has (4096 bytes), have you tried any bigger values for the threshold
> ?
> Because i'm still expecting less l1d cache misses and execution times the
> more we increase the threshold (relatively to l1d cache size per core).
> As per my previous not-so-stable numbers 28KB wasn't too bad.
>
>
> Regards,
> Ayoub
>

Ah, thanks for the prod, Ayoub. You are correct: The results in my previous
e-mail for the 0001-COPY-from-SIMD-v3-with-line_buf-periodic-refill.patch
patch are with LINE_BUF_FLUSH_AFTER set to its default of 4096. I will try
to measure what happens for larger LINE_BUF_FLUSH_AFTER values, hopefully
some time this week.
Best,
-Manni
-- 
-- Manni Wood EDB: https://www.enterprisedb.com

view thread (9+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
  In-Reply-To: <CAKWEB6r1a3yacHx8bAM3qfpUps4=rm+uUC7JxBfx3P5J0r_SdA@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox