agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Nathan Bossart <nathandbossart@gmail.com>
To: KAZAR Ayoub <ma_kazar@esi.dz>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>
Cc: ants.aasma@cybertec.at <ants.aasma@cybertec.at>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Shinya Kato <shinya11.kato@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
Date: Tue, 21 Oct 2025 13:55:07 -0500
Message-ID: <aPfXC3G30qpivfD9@nathan> (raw)
In-Reply-To: <CA+K2RumH-b=3-v0rfQ-oAbuQFxY8JLSSpVhmaJn+gRnX3t1_vg@mail.gmail.com>
References: <CA+K2RumC79NwWxBdofHOYo8SCSs0YCJic05Du=xOszRmoPf9FA@mail.gmail.com>
<CAN55FZ0houfWHn8_MEEefhprZvc33jr07GrBYo+Bp2yw=TVnKA@mail.gmail.com>
<CA+K2Ru=jHuz_Wpgar4Sobtxeb33qxx=o59ToOhZ=vpmkMqErnA@mail.gmail.com>
<CAN55FZ1J+6eM=F5GreWEBMJcNV_gifYyYY1b6xpYzun=nWPhMQ@mail.gmail.com>
<CAN55FZ109W90Ux_EBEqkkU2TyNqBNhdhN_1XPRGo3iiZ2L9b=A@mail.gmail.com>
<8615c983-1662-43b4-b0c9-49d194ac33aa@dunslane.net>
<CAN55FZ1KF7XNpm2XyG=M-sFUODai=6Z8a11xE3s4YRBeBKY3tA@mail.gmail.com>
<CA+K2RunFNDMxCWMX3PFSBa_r6REVwfEekaKHwg1C8KYYGePsnA@mail.gmail.com>
<CAN55FZ3e31ddFyf7XHW5G3ytuQwcXpetsb3wkx6q9oSp_zekhQ@mail.gmail.com>
<CA+K2RumH-b=3-v0rfQ-oAbuQFxY8JLSSpVhmaJn+gRnX3t1_vg@mail.gmail.com>
On Tue, Oct 21, 2025 at 08:17:01AM +0200, KAZAR Ayoub wrote:
>>> I'm also trying the idea of doing SIMD inside quotes with prefix XOR
>>> using carry less multiplication avoiding the slow path in all cases even
>>> with weird looking input, but it needs to take into consideration the
>>> availability of PCLMULQDQ instruction set with <wmmintrin.h> and here we
>>> go, it quickly starts to become dirty OR we can wait for the decision to
>>> start requiring x86-64-v2 or v3 which has SSE4.2 and AVX2.
>
> [...]
>
> Currently we are at 200-400Mbps which isn't that terrible compared to
> production and non production grade parsers (of course we don't only parse
> in our case), also we are using SSE2 only so theoretically if we add
> support for avx later on we'll have even better numbers.
> Maybe more micro optimizations to the current heuristic can squeeze it more.
I'd greatly prefer that we stick with SSE2/Neon (i.e., simd.h) unless the
gains are extraordinary. Beyond the inherent complexity of using
architecture-specific intrinsics, you also have to deal with configure-time
checks, runtime checks, and function pointer overhead juggling. That tends
to be a lot of work for the amount of gain.
--
nathan
view thread (178+ messages) latest in thread
Message-ID: <aPfXC3G30qpivfD9@nathan>
Permalink: ../aPfXC3G30qpivfD9@nathan/
Also on: postgresql.org/message-id/aPfXC3G30qpivfD9@nathan
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, ma_kazar@esi.dz, byavuz81@gmail.com, ants.aasma@cybertec.at, andrew@dunslane.net, shinya11.kato@gmail.com
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
In-Reply-To: <aPfXC3G30qpivfD9@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox