From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Jeff Davis <pgsql@j-davis.com>
Cc: Noah Misch <noah@leadboat.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Decision by Monday: PQescapeString() vs. encoding violation
Date: Sat, 15 Feb 2025 15:52:01 -0500
Message-ID: <243481.1739652721@sss.pgh.pa.us> (raw)
In-Reply-To: <5paqlw7soc66ud5ny3uq27qeeufo2pcm276rvv75hjhvns2sme@bvbqnxivxb2m>
References: <20250215012712.45@rfd.leadboat.com>
<186954494c6b0bf643b1aa42fc67e9e25386ebe0.camel@j-davis.com>
<5paqlw7soc66ud5ny3uq27qeeufo2pcm276rvv75hjhvns2sme@bvbqnxivxb2m>
Andres Freund <andres@anarazel.de> writes:
> On 2025-02-15 12:35:45 -0800, Jeff Davis wrote:
>> I am not suggesting a change, but there's a minor point about the
>> behavior of the replacement that I'd like to highlight:
>> Unicode discusses a choice[1]: "An ill-formed subsequence consisting of
>> more than one code unit could be treated as a single error or as
>> multiple errors."
> It seems completely infeasible to me to to implement the "single error"
> approach in a minor version. It'd afaict require non-trivial new
> infrastructure. We can't just consume up to the next byte without a high bit,
> because some encodings have subsequent bytes that are not guaranteed to have a
> high bit set.
Yeah. Also I think that probably depends on being able to tell the
difference between a first byte and a not-first byte of a multibyte
character, something that works in UTF-8 but not necessarily elsewhere.
As I commented in the security thread, Unicode's recommendations seem
pretty UTF-8-centric; I'm hesitant to adopt them wholesale in code
that has to deal with other encodings.
The v5 patch seems Good Enough(TM) to me. We can refine it later
perhaps; I don't think something like the above would affect
anything that external code should care about.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, andres@anarazel.de, pgsql@j-davis.com, noah@leadboat.com
Subject: Re: Decision by Monday: PQescapeString() vs. encoding violation
In-Reply-To: <243481.1739652721@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox