MIME-Version: 1.0
References: <CAN55FZ0J5iz9wFJLHcK7yNQqPb10_4ROoZiDu1wBZWSGC_fATg@mail.gmail.com>
 <CAKWEB6qY=mU62oAQFAVPCFWvwRuTPKBwxvM2aZ+J7p_9_MBmhQ@mail.gmail.com>
 <CAN55FZ2RPMxquXE6TH7dQkhtoiBcOOOZq8EOXj5COHv3ecP_cw@mail.gmail.com>
 <CA+K2Ru=fFTUVgEDr-fBed5aOMeDbH9vrOEhapXzHEpBeOxkucg@mail.gmail.com>
 <CAKWEB6pq7C0Wv1wT9Y1_c_1fn-+cR8pb210Pj3w2FcEOmNGxbQ@mail.gmail.com>
 <CAN55FZ2DT4-k06umn=7NYG+NoM6gnVJVQCCwRrr2qOraO+Jadw@mail.gmail.com>
 <aZikzQP6WPJ5Rq2S@nathan> <CAN55FZ3cBN_TncLVWyXAKm-KfewguN1AUjyRhoR6zL_QCxHh7A@mail.gmail.com>
 <CAKWEB6qzsZEQ4Czo9QBFiMXqdXVJknHUJwg6wjRwNzLn4+Jw0g@mail.gmail.com>
 <CAN55FZ2O2Ls==sdpROHqxWRx-PMBZ0riJ6eVKoHj8=vssTavxw@mail.gmail.com> <aZ3kYQnF9_u6sUQp@nathan>
In-Reply-To: <aZ3kYQnF9_u6sUQp@nathan>
From: Manni Wood <manni.wood@enterprisedb.com>
Date: Tue, 24 Feb 2026 22:06:13 -0600
Message-ID: <CAKWEB6oT5KbyF+uRRhjjJi7p2PmRdOzxp3T6vFcN04BCR-=B2w@mail.gmail.com>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>, KAZAR Ayoub <ma_kazar@esi.dz>, 
	Neil Conway <neil.conway@gmail.com>, Andrew Dunstan <andrew@dunslane.net>, 
	Shinya Kato <shinya11.kato@gmail.com>, 
	PostgreSQL-development <pgsql-hackers@postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000a9ccbb064b9e20e7"
Archived-At: <https://www.postgresql.org/message-id/CAKWEB6oT5KbyF%2BuRRhjjJi7p2PmRdOzxp3T6vFcN04BCR-%3DB2w%40mail.gmail.com>
Precedence: bulk

--000000000000a9ccbb064b9e20e7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 24, 2026 at 11:48=E2=80=AFAM Nathan Bossart <nathandbossart@gma=
il.com>
wrote:

> On Tue, Feb 24, 2026 at 04:57:21PM +0300, Nazir Bilal Yavuz wrote:
> > I will investigate this. However, please note that the current master
> > includes the inlining commit (dc592a4155), which makes the COPY FROM
> > faster. In my case,
> >
> > 1: current master without dc592a4155: 14400ms
> > 2: current master: 13960ms (%3 improvement against #1)
> > 3: current master + SIMD: 15123ms (%5 regression against #1 and %8
> > regression against #2)
> >
> > Is it possible for you to do a similar test? I mean dropping
> > dc592a4155 from the current master and re-running the benchmark, that
> > would be helpful.
>
> IMHO as long as the difference from v18 looks reasonable, commit-by-commi=
t
> regressions and improvements that even out in the end are okay.  That's
> perhaps a bit of mental gymnastics (e.g., what if we had committed the
> inlining patch for v18?), but I believe that's how we've dealt with simil=
ar
> problems in the past.  But maybe there are ways to avoid even these
> in-development regressions, too...
>
> --
> nathan
>

Oh yes, I see now.

Commit 18bcdb75 is just before the v9 patch got applied, so I used that as
"old master" and compared that with master (v9 applied) and then "master
(v9 applied) + v10 applied".

arm NARROW old master 18bcdb75
TXT :                 10997.568250 ms
CSV :                 10797.549000 ms
TXT with 1/3 escapes: 10299.047000 ms
CSV with 1/3 quotes:  12559.385750 ms

arm NARROW master (v9 applied)
TXT :                 10077.096250 ms  8.369778% improvement
CSV :                 10310.671250 ms  4.509151% improvement
TXT with 1/3 escapes: 9893.155000 ms  3.941064% improvement
CSV with 1/3 quotes:  12133.064750 ms  3.394441% improvement

arm NARROW v10
TXT :                 10467.816750 ms  4.816988% improvement
CSV :                 9986.288000 ms  7.513381% improvement
TXT with 1/3 escapes: 10323.173750 ms  -0.234262% regression
CSV with 1/3 quotes:  11843.611750 ms  5.699116% improvement


arm WIDE old master 18bcdb75
TXT :                 11825.771250 ms
CSV :                 13907.074000 ms
TXT with 1/3 escapes: 13430.691250 ms
CSV with 1/3 quotes:  17557.954500 ms

arm WIDE master (v9 applied)
TXT :                 10568.344750 ms  10.632934% improvement
CSV :                 13046.610500 ms  6.187236% improvement
TXT with 1/3 escapes: 12193.088500 ms  9.214736% improvement
CSV with 1/3 quotes:  16629.319000 ms  5.288973% improvement

arm WIDE v10
TXT :                 9064.959000 ms  23.345727% improvement
CSV :                 9019.553250 ms  35.144134% improvement
TXT with 1/3 escapes: 12344.497250 ms  8.087402% improvement
CSV with 1/3 quotes:  15495.863750 ms  11.744482% improvement


x86 NARROW old master 18bcdb75
TXT :                 25909.060500 ms
CSV :                 28137.591250 ms
TXT with 1/3 escapes: 27794.177000 ms
CSV with 1/3 quotes:  34541.704750 ms

x86 NARROW master
TXT :                 26359.319000 ms  -1.737842% regression
CSV :                 25661.199750 ms  8.801007% improvement
TXT with 1/3 escapes: 28170.085250 ms  -1.352471% regression
CSV with 1/3 quotes:  32638.147500 ms  5.510895% improvement

x86 NARROW v10
TXT :                 26416.331500 ms  -1.957890% regression
CSV :                 25318.727500 ms  10.018142% improvement
TXT with 1/3 escapes: 28608.007500 ms  -2.928061% regression
CSV with 1/3 quotes:  32805.627750 ms  5.026032% improvement

x86 WIDE old master 18bcdb75
TXT :                 28778.426500 ms
CSV :                 35671.908000 ms
TXT with 1/3 escapes: 32441.549750 ms
CSV with 1/3 quotes:  47024.416000 ms

x86 WIDE master
TXT :                 26475.164250 ms  8.003434% improvement
CSV :                 31963.478500 ms  10.395938% improvement
TXT with 1/3 escapes: 29671.120750 ms  8.539755% improvement
CSV with 1/3 quotes:  40391.616250 ms  14.105012% improvement

x86 WIDE v10
TXT :                 23067.046750 ms  19.846046% improvement
CSV :                 23259.092250 ms  34.797174% improvement
TXT with 1/3 escapes: 31796.098250 ms  1.989583% improvement
CSV with 1/3 quotes:  42925.792250 ms  8.715948% improvement

--=20
-- Manni Wood EDB: https://www.enterprisedb.com

--000000000000a9ccbb064b9e20e7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote g=
mail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Feb 24,=
 2026 at 11:48=E2=80=AFAM Nathan Bossart &lt;<a href=3D"mailto:nathandbossa=
rt@gmail.com">nathandbossart@gmail.com</a>&gt; wrote:<br></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex">On Tue, Feb 24, 2026 at 04:57:21PM +0=
300, Nazir Bilal Yavuz wrote:<br>
&gt; I will investigate this. However, please note that the current master<=
br>
&gt; includes the inlining commit (dc592a4155), which makes the COPY FROM<b=
r>
&gt; faster. In my case,<br>
&gt; <br>
&gt; 1: current master without dc592a4155: 14400ms<br>
&gt; 2: current master: 13960ms (%3 improvement against #1)<br>
&gt; 3: current master + SIMD: 15123ms (%5 regression against #1 and %8<br>
&gt; regression against #2)<br>
&gt; <br>
&gt; Is it possible for you to do a similar test? I mean dropping<br>
&gt; dc592a4155 from the current master and re-running the benchmark, that<=
br>
&gt; would be helpful.<br>
<br>
IMHO as long as the difference from v18 looks reasonable, commit-by-commit<=
br>
regressions and improvements that even out in the end are okay.=C2=A0 That&=
#39;s<br>
perhaps a bit of mental gymnastics (e.g., what if we had committed the<br>
inlining patch for v18?), but I believe that&#39;s how we&#39;ve dealt with=
 similar<br>
problems in the past.=C2=A0 But maybe there are ways to avoid even these<br=
>
in-development regressions, too...<br>
<br>
-- <br>
nathan<br>
</blockquote></div><div><br clear=3D"all"></div><div>Oh yes, I see now.</di=
v><div><br></div><div>Commit=C2=A018bcdb75 is just before the v9 patch got =
applied, so I used that as &quot;old master&quot; and compared that with ma=
ster (v9 applied) and then &quot;master (v9 applied)=C2=A0+ v10 applied&quo=
t;.=C2=A0</div><div><br></div><div>arm NARROW old master 18bcdb75<br>TXT : =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10997.568250 ms<br>=
CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10797.549000 =
ms<br>TXT with 1/3 escapes: 10299.047000 ms<br>CSV with 1/3 quotes: =C2=A01=
2559.385750 ms<br><br>arm NARROW master (v9 applied)<br>TXT : =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10077.096250 ms =C2=A08.369778% =
improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 10310.671250 ms =C2=A04.509151% improvement<br>TXT with 1/3 escapes: 98=
93.155000 ms =C2=A03.941064% improvement<br>CSV with 1/3 quotes: =C2=A01213=
3.064750 ms =C2=A03.394441% improvement<br><br>arm NARROW v10<br>TXT : =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10467.816750 ms =C2=A0=
4.816988% improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 9986.288000 ms =C2=A07.513381% improvement<br>TXT with 1/3 es=
capes: 10323.173750 ms =C2=A0-0.234262% regression<br>CSV with 1/3 quotes: =
=C2=A011843.611750 ms =C2=A05.699116% improvement<br><br><br>arm WIDE old m=
aster 18bcdb75<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 11825.771250 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 13907.074000 ms<br>TXT with 1/3 escapes: 13430.691250 ms<br>C=
SV with 1/3 quotes: =C2=A017557.954500 ms<br><br>arm WIDE master (v9 applie=
d)<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10568.3=
44750 ms =C2=A010.632934% improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 13046.610500 ms =C2=A06.187236% improvement<br>=
TXT with 1/3 escapes: 12193.088500 ms =C2=A09.214736% improvement<br>CSV wi=
th 1/3 quotes: =C2=A016629.319000 ms =C2=A05.288973% improvement<br><br>arm=
 WIDE v10<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
9064.959000 ms =C2=A023.345727% improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 9019.553250 ms =C2=A035.144134% improvem=
ent<br>TXT with 1/3 escapes: 12344.497250 ms =C2=A08.087402% improvement<br=
>CSV with 1/3 quotes: =C2=A015495.863750 ms =C2=A011.744482% improvement<br=
><br><br><br>x86 NARROW old master 18bcdb75<br>TXT : =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 25909.060500 ms<br>CSV : =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 28137.591250 ms<br>TXT with 1/3 e=
scapes: 27794.177000 ms<br>CSV with 1/3 quotes: =C2=A034541.704750 ms<br><b=
r>x86 NARROW master<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 26359.319000 ms =C2=A0-1.737842% regression<br>CSV : =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 25661.199750 ms =C2=A08.80100=
7% improvement<br>TXT with 1/3 escapes: 28170.085250 ms =C2=A0-1.352471% re=
gression<br>CSV with 1/3 quotes: =C2=A032638.147500 ms =C2=A05.510895% impr=
ovement<br><br>x86 NARROW v10<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 26416.331500 ms =C2=A0-1.957890% regression<br>CSV : =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 25318.727500 ms =C2=
=A010.018142% improvement<br>TXT with 1/3 escapes: 28608.007500 ms =C2=A0-2=
.928061% regression<br>CSV with 1/3 quotes: =C2=A032805.627750 ms =C2=A05.0=
26032% improvement<br><br>x86 WIDE old master 18bcdb75<br>TXT : =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 28778.426500 ms<br>CSV : =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 35671.908000 ms<br>TXT=
 with 1/3 escapes: 32441.549750 ms<br>CSV with 1/3 quotes: =C2=A047024.4160=
00 ms<br><br>x86 WIDE master<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 26475.164250 ms =C2=A08.003434% improvement<br>CSV : =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 31963.478500 ms =C2=
=A010.395938% improvement<br>TXT with 1/3 escapes: 29671.120750 ms =C2=A08.=
539755% improvement<br>CSV with 1/3 quotes: =C2=A040391.616250 ms =C2=A014.=
105012% improvement<br><br>x86 WIDE v10<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 23067.046750 ms =C2=A019.846046% improvemen=
t<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 23259.09=
2250 ms =C2=A034.797174% improvement<br>TXT with 1/3 escapes: 31796.098250 =
ms =C2=A01.989583% improvement<br>CSV with 1/3 quotes: =C2=A042925.792250 m=
s =C2=A08.715948% improvement</div><div><br></div><span class=3D"gmail_sign=
ature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div=
 dir=3D"ltr"><span style=3D"color:rgb(29,28,29);font-family:Monaco,Menlo,Co=
nsolas,&quot;Courier New&quot;,monospace;font-size:12px;background-color:rg=
ba(29,28,29,0.04)">--
Manni Wood
EDB: </span><a href=3D"https://www.enterprisedb.com/" rel=3D"noopener noref=
errer" style=3D"font-family:Monaco,Menlo,Consolas,&quot;Courier New&quot;,m=
onospace;font-size:12px;background-color:rgba(29,28,29,0.04)" target=3D"_bl=
ank">https://www.enterprisedb.com</a></div></div></div>

--000000000000a9ccbb064b9e20e7--