Date: Tue, 8 Sep 2026 15:58:25 -0500
From: Nathan Bossart <nathandbossart@gmail.com>
To: Greg Burd <greg@burd.me>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>, KAZAR Ayoub <ma_kazar@esi.dz>,
	Neil Conway <neil.conway@gmail.com>,
	Andrew Dunstan <andrew@dunslane.net>,
	Shinya Kato <shinya11.kato@gmail.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
Message-ID: <aqB28YZZHC2xnZMq@nathan>
References: <apWOr1kgVSqweBnU@nathan>
 <CAN55FZ1utgaKb1DbW7tr+2=s4Hm1yiCEeNDY9m=qP_M_drafCg@mail.gmail.com>
 <78e736c8-7f4e-46c2-9fcf-a6839e393d64@app.fastmail.com>
 <CAN55FZ0UJg+9wcyTCUG656c9DKCNvTGVuuVe4EMMVCHPwGS4aQ@mail.gmail.com>
 <apcmTvBmA27RoyYL@nathan>
 <CAN55FZ1qFb4Yo3-MWeQfGQnu1_Ksx4xcFC2m3hyw8a--+eHQYQ@mail.gmail.com>
 <CAN55FZ3U2waD=5ZuVmju-HtZ4cO4onxHN4jXskk8Y11M6o3NOQ@mail.gmail.com>
 <9434f7bb-5865-4828-9f23-1dd30e50aaac@app.fastmail.com>
 <aqArgOhtHfM2_iJe@nathan>
 <633c2a60-e5b9-49d2-a93d-946d184d946d@app.fastmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <633c2a60-e5b9-49d2-a93d-946d184d946d@app.fastmail.com>
Archived-At: <https://www.postgresql.org/message-id/aqB28YZZHC2xnZMq%40nathan>
Precedence: bulk

On Tue, Sep 08, 2026 at 11:58:42AM -0400, Greg Burd wrote:
> The test writes one full input buffer (INPUT_BUF_SIZE, 65536 bytes) down a
> COPY FROM PROGRAM pipe: a 65524-byte line, then a complete \. marker, then
> 8 trailing bytes. That leaves 11 bytes unexamined when the SIMD path wants
> to refill, which is under sizeof(Vector8) and already holds the whole
> marker. The writer then dribbles a byte every 50ms and keeps the pipe open,
> so a speculative load blocks rather than seeing EOF. statement_timeout
> bounds the failure, so a regression fails the test instead of hanging the
> suite. Skipped on Windows, where the blocking pipe and SIGPIPE behavior
> isn't reliable.

I'm not sure this particular corner case deserves its own TAP test.  IMHO
that is disproportionately expensive.  It really only serves to prevent us
from removing the backslash scan from the SIMD path, which I don't think
we're likely to do.  Furthermore, I'm not aware of any existing examples
where a test writes and then invokes its own Perl script, and I can't get
too excited about being the first to commit something like that.  So, I'd
prefer to leave this test out for now.  If the new test fit cleanly into an
existing suite or was less magical, I might be more eager to include it,
but it's probably still unlikely to be worth the energy and maintenance
risk.

-- 
nathan