agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Nathan Bossart <nathandbossart@gmail.com>
To: Greg Burd <greg@burd.me>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>
Cc: KAZAR Ayoub <ma_kazar@esi.dz>
Cc: Neil Conway <neil.conway@gmail.com>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Shinya Kato <shinya11.kato@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
Date: Tue, 8 Sep 2026 15:58:25 -0500
Message-ID: <aqB28YZZHC2xnZMq@nathan> (raw)
In-Reply-To: <633c2a60-e5b9-49d2-a93d-946d184d946d@app.fastmail.com>
References: <apWOr1kgVSqweBnU@nathan>
<CAN55FZ1utgaKb1DbW7tr+2=s4Hm1yiCEeNDY9m=qP_M_drafCg@mail.gmail.com>
<78e736c8-7f4e-46c2-9fcf-a6839e393d64@app.fastmail.com>
<CAN55FZ0UJg+9wcyTCUG656c9DKCNvTGVuuVe4EMMVCHPwGS4aQ@mail.gmail.com>
<apcmTvBmA27RoyYL@nathan>
<CAN55FZ1qFb4Yo3-MWeQfGQnu1_Ksx4xcFC2m3hyw8a--+eHQYQ@mail.gmail.com>
<CAN55FZ3U2waD=5ZuVmju-HtZ4cO4onxHN4jXskk8Y11M6o3NOQ@mail.gmail.com>
<9434f7bb-5865-4828-9f23-1dd30e50aaac@app.fastmail.com>
<aqArgOhtHfM2_iJe@nathan>
<633c2a60-e5b9-49d2-a93d-946d184d946d@app.fastmail.com>
On Tue, Sep 08, 2026 at 11:58:42AM -0400, Greg Burd wrote:
> The test writes one full input buffer (INPUT_BUF_SIZE, 65536 bytes) down a
> COPY FROM PROGRAM pipe: a 65524-byte line, then a complete \. marker, then
> 8 trailing bytes. That leaves 11 bytes unexamined when the SIMD path wants
> to refill, which is under sizeof(Vector8) and already holds the whole
> marker. The writer then dribbles a byte every 50ms and keeps the pipe open,
> so a speculative load blocks rather than seeing EOF. statement_timeout
> bounds the failure, so a regression fails the test instead of hanging the
> suite. Skipped on Windows, where the blocking pipe and SIGPIPE behavior
> isn't reliable.
I'm not sure this particular corner case deserves its own TAP test. IMHO
that is disproportionately expensive. It really only serves to prevent us
from removing the backslash scan from the SIMD path, which I don't think
we're likely to do. Furthermore, I'm not aware of any existing examples
where a test writes and then invokes its own Perl script, and I can't get
too excited about being the first to commit something like that. So, I'd
prefer to leave this test out for now. If the new test fit cleanly into an
existing suite or was less magical, I might be more eager to include it,
but it's probably still unlikely to be worth the energy and maintenance
risk.
--
nathan
view thread (178+ messages) latest in thread
Message-ID: <aqB28YZZHC2xnZMq@nathan>
Permalink: ../aqB28YZZHC2xnZMq@nathan/
Also on: postgresql.org/message-id/aqB28YZZHC2xnZMq@nathan
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, greg@burd.me, byavuz81@gmail.com, ma_kazar@esi.dz, neil.conway@gmail.com, andrew@dunslane.net, shinya11.kato@gmail.com
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
In-Reply-To: <aqB28YZZHC2xnZMq@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox