MIME-Version: 1.0
References: 
 <CA+1gyqJJJDhq=cc_D0ad59WH_OD2G_mN54xTru0KYoNaLkF48Q@mail.gmail.com>
In-Reply-To: 
 <CA+1gyqJJJDhq=cc_D0ad59WH_OD2G_mN54xTru0KYoNaLkF48Q@mail.gmail.com>
From: Zhongpu Chen <chenloveit@gmail.com>
Date: Sat, 2 May 2026 10:39:26 +0800
Message-ID: 
 <CA+1gyq+LF_91g_i0WXeKK6JGF8viaqaF213S-9Arq=SG=4GAaA@mail.gmail.com>
Subject: Re: Proposal: tighten validation for legacy EUC encodings or document
 that accepted byte sequences may be unconvertible to UTF8
To: pgsql-hackers@lists.postgresql.org
Content-Type: multipart/alternative; boundary="000000000000dd603f0650cc9be7"
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2B1gyq%2BLF_91g_i0WXeKK6JGF8viaqaF213S-9Arq%3DSG%3D4GAaA%40mail.gmail.com>
Precedence: bulk

--000000000000dd603f0650cc9be7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

The issue is not specific to E'\\x..' literals. A normal COPY FROM data
file with ENCODING 'EUC_CN' can create text rows that later cannot be
retrieved with SELECT.

 This suggests that input validation for EUC_CN is only structural, while
the EUC_CN-to-UTF8 conversion table is stricter.


On Sat, May 2, 2026 at 10:31=E2=80=AFAM Zhongpu Chen <chenloveit@gmail.com>=
 wrote:

> See the related bug report
> https://www.postgresql.org/message-id/CA%2B1gyqL7uiQhfLcYWpHNUKQgHjQc7sOP=
thSTiaxLDZzcrGFYSg%40mail.gmail.com
>
> Currently PostgreSQL accepts structurally well-formed EUC_CN byte
> sequences such as 0xA2A3 into text columns. The value round-trips when
> client_encoding is EUC_CN, but fails when client_encoding is UTF8 because
> euc_cn_to_utf8 has no mapping.
>
> If this behavior is intentional for compatibility, the documentation
> should explicitly say that validation for some legacy encodings is
> byte-structure validation, not mapping-table validation.
> If it is not intentional, stricter validation could reject unassigned byt=
e
> positions at input time.
>
> --
> Zhongpu Chen
>


--=20
Zhongpu Chen

--000000000000dd603f0650cc9be7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>The issue is not specific to E&#39;\\x..&#39; literal=
s. A normal COPY FROM data file with ENCODING &#39;EUC_CN&#39; can create t=
ext rows that later cannot be retrieved with SELECT.</div><div><br></div><d=
iv>=C2=A0This suggests that input validation for EUC_CN is only structural,=
 while the EUC_CN-to-UTF8 conversion table is stricter.</div><br></div><br>=
<div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"=
gmail_attr">On Sat, May 2, 2026 at 10:31=E2=80=AFAM Zhongpu Chen &lt;<a hre=
f=3D"mailto:chenloveit@gmail.com">chenloveit@gmail.com</a>&gt; wrote:<br></=
div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div=
>See the related bug report=C2=A0<a href=3D"https://www.postgresql.org/mess=
age-id/CA%2B1gyqL7uiQhfLcYWpHNUKQgHjQc7sOPthSTiaxLDZzcrGFYSg%40mail.gmail.c=
om" target=3D"_blank">https://www.postgresql.org/message-id/CA%2B1gyqL7uiQh=
fLcYWpHNUKQgHjQc7sOPthSTiaxLDZzcrGFYSg%40mail.gmail.com</a></div><div><br><=
/div><div>Currently PostgreSQL accepts structurally well-formed EUC_CN byte=
 sequences such as 0xA2A3 into text columns. The value round-trips when cli=
ent_encoding is EUC_CN, but fails when client_encoding is UTF8 because euc_=
cn_to_utf8 has no mapping.<br><br></div><div>If this behavior is intentiona=
l for compatibility, the documentation should explicitly say that validatio=
n for some legacy encodings is byte-structure validation, not mapping-table=
 validation.<br>If it is not intentional, stricter validation could reject =
unassigned byte positions at input time.</div><br><span class=3D"gmail_sign=
ature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div=
 dir=3D"ltr">Zhongpu Chen</div></div></div>
</blockquote></div><div><br clear=3D"all"></div><br><span class=3D"gmail_si=
gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d=
iv dir=3D"ltr">Zhongpu Chen</div></div>

--000000000000dd603f0650cc9be7--