MIME-Version: 1.0
References: <aa8QlTVEDhG1JU0Z@nathan>
 <CAN55FZ08kqmA+B9pzPDy-QstxAd=cK-RqjbR3cWBjPF_8-FXAw@mail.gmail.com>
 <abBQbdFa6OsG8TGu@nathan>
 <CAN55FZ3jXs7XDsP_-v_jUBquRu4uAdheN3xcmW=WhAyKwFLSjg@mail.gmail.com>
 <abGv0ScUWVa6eogw@nathan>
 <CAN55FZ3gdK8dGrEo0M6KFW97OaF8TUbjO_dFoxQKi63davE-jA@mail.gmail.com>
 <abG8R6HkOHyUuyWb@nathan>
 <CAN55FZ0yfETy4UEA5rOJ9S06JSOtiWF8TW_+yi3yjVAcrLqKLA@mail.gmail.com>
 <abHTvkeIK37hj9oS@nathan>
 <CAN55FZ1sn-2tVX_n9C5UNBCfDPjSDOCT4zkMeDsB7MaZ9SUBTw@mail.gmail.com>
 <abL5ylYjmopRRmvy@nathan>
In-Reply-To: <abL5ylYjmopRRmvy@nathan>
From: Manni Wood <manni.wood@enterprisedb.com>
Date: Thu, 12 Mar 2026 21:39:38 -0500
Message-ID: 
 <CAKWEB6pMbdMDvhfaX1Z0eSULVQFYhEhssaRHdOxAX_5OYubxKw@mail.gmail.com>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>, KAZAR Ayoub <ma_kazar@esi.dz>,
	Neil Conway <neil.conway@gmail.com>, Andrew Dunstan <andrew@dunslane.net>,
	Shinya Kato <shinya11.kato@gmail.com>,
	PostgreSQL-development <pgsql-hackers@postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000008a7d5f064cdec830"
Archived-At: 
 <https://www.postgresql.org/message-id/CAKWEB6pMbdMDvhfaX1Z0eSULVQFYhEhssaRHdOxAX_5OYubxKw%40mail.gmail.com>
Precedence: bulk

--0000000000008a7d5f064cdec830
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 12, 2026 at 12:37=E2=80=AFPM Nathan Bossart <nathandbossart@gma=
il.com>
wrote:

> Here is what I have staged for commit, which I'm planning to do tomorrow.
> Please review and/or test if you are able.
>
> --
> nathan
>

Hello, Nathan!

I found some time this evening to run some benchmarks using your v15 patch.
I hope these help.

lz4 - arm

arm NARROW master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D lz4
TXT :                 10203.493250 ms
CSV :                 10217.946000 ms
TXT with 1/3 escapes: 10305.912750 ms
CSV with 1/3 quotes:  12339.182000 ms

arm NARROW v15 default_toast_compression =3D lz4
TXT :                 10205.261500 ms  -0.017330% regression
CSV :                 10358.898500 ms  -1.379460% regression
TXT with 1/3 escapes: 10053.073000 ms  2.453347% improvement
CSV with 1/3 quotes:  11881.337000 ms  3.710497% improvement

arm WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D lz4
TXT :                 5613.525250 ms
CSV :                 8069.692750 ms
TXT with 1/3 escapes: 7088.888250 ms
CSV with 1/3 quotes:  10902.545500 ms

arm WIDE v15 default_toast_compression =3D lz4
TXT :                 3201.494500 ms  42.968200% improvement
CSV :                 3146.033750 ms  61.014207% improvement
TXT with 1/3 escapes: 6677.907500 ms  5.797535% improvement
CSV with 1/3 quotes:  10766.909500 ms  1.244076% improvement

lz4 - x86

x86 NARROW master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D lz4
TXT :                 26110.287750 ms
CSV :                 27923.199750 ms
TXT with 1/3 escapes: 27984.483250 ms
CSV with 1/3 quotes:  34387.239000 ms

x86 NARROW v15 default_toast_compression =3D lz4
TXT :                 26019.629000 ms  0.347215% improvement
CSV :                 26379.889000 ms  5.526984% improvement
TXT with 1/3 escapes: 28865.322750 ms  -3.147600% regression
CSV with 1/3 quotes:  33218.293250 ms  3.399359% improvement

x86 WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D lz4
TXT :                 15829.765000 ms
CSV :                 20479.146000 ms
TXT with 1/3 escapes: 18437.507500 ms
CSV with 1/3 quotes:  29749.379250 ms

x86 WIDE v15 default_toast_compression =3D lz4
TXT :                 8056.305000 ms  49.106604% improvement
CSV :                 7997.555500 ms  60.947808% improvement
TXT with 1/3 escapes: 16324.925500 ms  11.458067% improvement
CSV with 1/3 quotes:  29978.346500 ms  -0.769654% regression


pglz - arm

arm NARROW master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D pglz
TXT :                 10334.666250 ms
CSV :                 10978.851250 ms
TXT with 1/3 escapes: 11076.502750 ms
CSV with 1/3 quotes:  12582.679000 ms

arm NARROW v15 default_toast_compression =3D pglz
TXT :                 10002.507750 ms  3.214023% improvement
CSV :                 10017.436250 ms  8.756973% improvement
TXT with 1/3 escapes: 10179.949000 ms  8.094195% improvement
CSV with 1/3 quotes:  12088.836750 ms  3.924778% improvement

arm WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D pglz
TXT :                 11403.206000 ms
CSV :                 13915.718750 ms
TXT with 1/3 escapes: 12888.060250 ms
CSV with 1/3 quotes:  16741.463000 ms

arm WIDE v15 default_toast_compression =3D pglz
TXT :                 9005.868250 ms  21.023366% improvement
CSV :                 8935.159250 ms  35.790889% improvement
TXT with 1/3 escapes: 12432.655250 ms  3.533542% improvement
CSV with 1/3 quotes:  16564.852250 ms  1.054930% improvement

pglz - x86

x86 NARROW master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D pglz
TXT :                 26404.516250 ms
CSV :                 28138.719000 ms
TXT with 1/3 escapes: 28084.379750 ms
CSV with 1/3 quotes:  34502.702250 ms

x86 NARROW v15 default_toast_compression =3D pglz
TXT :                 26438.415000 ms  -0.128382% regression
CSV :                 26869.718000 ms  4.509804% improvement
TXT with 1/3 escapes: 29379.299750 ms  -4.610819% regression
CSV with 1/3 quotes:  33371.390250 ms  3.278908% improvement

x86 WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235)
default_toast_compression =3D pglz
TXT :                 30595.372000 ms
CSV :                 35665.908500 ms
TXT with 1/3 escapes: 32746.252000 ms
CSV with 1/3 quotes:  44136.542750 ms

x86 WIDE v15 default_toast_compression =3D pglz
TXT :                 22681.770750 ms  25.865354% improvement
CSV :                 22692.153000 ms  36.375789% improvement
TXT with 1/3 escapes: 30638.978000 ms  6.435161% improvement
CSV with 1/3 quotes:  44330.233000 ms  -0.438843% regression
--=20
-- Manni Wood EDB: https://www.enterprisedb.com

--0000000000008a7d5f064cdec830
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div><br></div><div><br></div></div><br><=
div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"g=
mail_attr">On Thu, Mar 12, 2026 at 12:37=E2=80=AFPM Nathan Bossart &lt;<a h=
ref=3D"mailto:nathandbossart@gmail.com">nathandbossart@gmail.com</a>&gt; wr=
ote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px=
 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Here is wha=
t I have staged for commit, which I&#39;m planning to do tomorrow.<br>
Please review and/or test if you are able.<br>
<br>
-- <br>
nathan<br>
</blockquote></div><div><br clear=3D"all"></div><div>Hello, Nathan!<br><br>=
I found some time this evening to run some benchmarks using your v15 patch.=
 I hope these help.<br><br>lz4 - arm<br><br>arm NARROW master (git revert d=
c592a41557b072178f1798700bf9c69cd8e4235) default_toast_compression =3D lz4<=
br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10203.4932=
50 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1021=
7.946000 ms<br>TXT with 1/3 escapes: 10305.912750 ms<br>CSV with 1/3 quotes=
: =C2=A012339.182000 ms<br><br>arm NARROW v15 default_toast_compression =3D=
 lz4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10205=
.261500 ms =C2=A0-0.017330% regression<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10358.898500 ms =C2=A0-1.379460% regression<br=
>TXT with 1/3 escapes: 10053.073000 ms =C2=A02.453347% improvement<br>CSV w=
ith 1/3 quotes: =C2=A011881.337000 ms =C2=A03.710497% improvement<br><br>ar=
m WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235) default=
_toast_compression =3D lz4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 5613.525250 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 8069.692750 ms<br>TXT with 1/3 escapes: 7088.88825=
0 ms<br>CSV with 1/3 quotes: =C2=A010902.545500 ms<br><br>arm WIDE v15 defa=
ult_toast_compression =3D lz4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 3201.494500 ms =C2=A042.968200% improvement<br>CSV : =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 3146.033750 ms =C2=
=A061.014207% improvement<br>TXT with 1/3 escapes: 6677.907500 ms =C2=A05.7=
97535% improvement<br>CSV with 1/3 quotes: =C2=A010766.909500 ms =C2=A01.24=
4076% improvement<br><br>lz4 - x86<br><br>x86 NARROW master (git revert dc5=
92a41557b072178f1798700bf9c69cd8e4235) default_toast_compression =3D lz4<br=
>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 26110.287750=
 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 27923.=
199750 ms<br>TXT with 1/3 escapes: 27984.483250 ms<br>CSV with 1/3 quotes: =
=C2=A034387.239000 ms<br><br>x86 NARROW v15 default_toast_compression =3D l=
z4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 26019.6=
29000 ms =C2=A00.347215% improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 26379.889000 ms =C2=A05.526984% improvement<br>=
TXT with 1/3 escapes: 28865.322750 ms =C2=A0-3.147600% regression<br>CSV wi=
th 1/3 quotes: =C2=A033218.293250 ms =C2=A03.399359% improvement<br><br>x86=
 WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e4235) default_=
toast_compression =3D lz4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 15829.765000 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 20479.146000 ms<br>TXT with 1/3 escapes: 18437.507=
500 ms<br>CSV with 1/3 quotes: =C2=A029749.379250 ms<br><br>x86 WIDE v15 de=
fault_toast_compression =3D lz4<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 8056.305000 ms =C2=A049.106604% improvement<br>CSV : =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 7997.555500 ms =C2=
=A060.947808% improvement<br>TXT with 1/3 escapes: 16324.925500 ms =C2=A011=
.458067% improvement<br>CSV with 1/3 quotes: =C2=A029978.346500 ms =C2=A0-0=
.769654% regression<br><br><br><br>pglz - arm<br><br>arm NARROW master (git=
 revert dc592a41557b072178f1798700bf9c69cd8e4235) default_toast_compression=
 =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
10334.666250 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 10978.851250 ms<br>TXT with 1/3 escapes: 11076.502750 ms<br>CSV with=
 1/3 quotes: =C2=A012582.679000 ms<br><br>arm NARROW v15 default_toast_comp=
ression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 10002.507750 ms =C2=A03.214023% improvement<br>CSV : =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 10017.436250 ms =C2=A08.756973% i=
mprovement<br>TXT with 1/3 escapes: 10179.949000 ms =C2=A08.094195% improve=
ment<br>CSV with 1/3 quotes: =C2=A012088.836750 ms =C2=A03.924778% improvem=
ent<br><br>arm WIDE master (git revert dc592a41557b072178f1798700bf9c69cd8e=
4235) default_toast_compression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 11403.206000 ms<br>CSV : =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 13915.718750 ms<br>TXT with 1/3 esca=
pes: 12888.060250 ms<br>CSV with 1/3 quotes: =C2=A016741.463000 ms<br><br>a=
rm WIDE v15 default_toast_compression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 9005.868250 ms =C2=A021.023366% impr=
ovement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 89=
35.159250 ms =C2=A035.790889% improvement<br>TXT with 1/3 escapes: 12432.65=
5250 ms =C2=A03.533542% improvement<br>CSV with 1/3 quotes: =C2=A016564.852=
250 ms =C2=A01.054930% improvement<br><br>pglz - x86<br><br>x86 NARROW mast=
er (git revert dc592a41557b072178f1798700bf9c69cd8e4235) default_toast_comp=
ression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 26404.516250 ms<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 28138.719000 ms<br>TXT with 1/3 escapes: 28084.379750 ms<br>C=
SV with 1/3 quotes: =C2=A034502.702250 ms<br><br>x86 NARROW v15 default_toa=
st_compression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 26438.415000 ms =C2=A0-0.128382% regression<br>CSV : =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 26869.718000 ms =C2=A04.50=
9804% improvement<br>TXT with 1/3 escapes: 29379.299750 ms =C2=A0-4.610819%=
 regression<br>CSV with 1/3 quotes: =C2=A033371.390250 ms =C2=A03.278908% i=
mprovement<br><br>x86 WIDE master (git revert dc592a41557b072178f1798700bf9=
c69cd8e4235) default_toast_compression =3D pglz<br>TXT : =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 30595.372000 ms<br>CSV : =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 35665.908500 ms<br>TXT with 1=
/3 escapes: 32746.252000 ms<br>CSV with 1/3 quotes: =C2=A044136.542750 ms<b=
r><br>x86 WIDE v15 default_toast_compression =3D pglz<br>TXT : =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 22681.770750 ms =C2=A025.8653=
54% improvement<br>CSV : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 22692.153000 ms =C2=A036.375789% improvement<br>TXT with 1/3 escapes=
: 30638.978000 ms =C2=A06.435161% improvement<br>CSV with 1/3 quotes: =C2=
=A044330.233000 ms =C2=A0-0.438843% regression</div><span class=3D"gmail_si=
gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d=
iv dir=3D"ltr"><span style=3D"color:rgb(29,28,29);font-family:Monaco,Menlo,=
Consolas,&quot;Courier New&quot;,monospace;font-size:12px;background-color:=
rgba(29,28,29,0.04)">--
Manni Wood
EDB: </span><a href=3D"https://www.enterprisedb.com/" rel=3D"noopener noref=
errer" style=3D"font-family:Monaco,Menlo,Consolas,&quot;Courier New&quot;,m=
onospace;font-size:12px;background-color:rgba(29,28,29,0.04)" target=3D"_bl=
ank">https://www.enterprisedb.com</a></div></div></div>

--0000000000008a7d5f064cdec830--