Message-ID: <464a14fcbee9567ace136716714b8c18f9bd87fb.camel@cybertec.at>
Subject: Re: BUG #19341: REPLACE() fails to match final character when using
 nondeterministic ICU collation
From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, adam.warland@infor.com,
	pgsql-bugs@lists.postgresql.org
Date: Tue, 02 Dec 2025 18:51:14 +0100
In-Reply-To: <375807.1764696352@sss.pgh.pa.us>
References: <19341-1d9a22915edfec58@postgresql.org>
	 <60b467ac47d5c6bbb9c21c3bdfecce8e5de60cda.camel@cybertec.at>
	 <6935ea1e-cfb1-400b-9057-3d43b55d621a@iki.fi>
	 <9f47c6b24d6cf59002603caf19b8b3f4854176be.camel@cybertec.at>
	 <375807.1764696352@sss.pgh.pa.us>
Content-Type: multipart/mixed; boundary="=-E8vSkERV/qAhr9y48Fjx"
User-Agent: Evolution 3.56.2 (3.56.2-2.fc42) 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/464a14fcbee9567ace136716714b8c18f9bd87fb.camel%40cybertec.at>
Precedence: bulk

--=-E8vSkERV/qAhr9y48Fjx
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2025-12-02 at 12:25 -0500, Tom Lane wrote:
> Laurenz Albe <laurenz.albe@cybertec.at> writes:
> > > > for (const char *test_end =3D hptr; test_end <=3D haystack_end; tes=
t_end +=3D pg_mblen(test_end))
>=20
> > During the last iteration of the loop, "test_end" will be equal to "hay=
stack_end",
> > and the loop increment will call "pg_mblen(test_end)".
>=20
> Right, clearly unsafe (and I bet valgrind would complain about it).
> You need to rearrange the loop logic so that we won't attempt to
> increment test_end that last time through.  Perhaps a for-loop
> isn't the best way to write it.

Right.  The attached patch v3 turns it into a while loop to avoid
the problem.

Yours,
Laurenz Albe

--=-E8vSkERV/qAhr9y48Fjx
Content-Disposition: attachment;
	filename*0=v3-0001-Fix-greedy-substring-search-for-non-deterministic.pat;
	filename*1=ch
Content-Transfer-Encoding: base64
Content-Type: text/x-patch;
	name="v3-0001-Fix-greedy-substring-search-for-non-deterministic.patch";
	charset="UTF-8"

RnJvbSA0MWM0NDFhMDJkYjM0MTk1Mjc2M2QwNGJiODIyYTYzNGI3MTFjNzM1IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBMYXVyZW56IEFsYmUgPGxhdXJlbnouYWxiZUBjeWJlcnRlYy5h
dD4KRGF0ZTogVHVlLCAyIERlYyAyMDI1IDE4OjQ5OjExICswMTAwClN1YmplY3Q6IFtQQVRDSCB2
M10gRml4IGdyZWVkeSBzdWJzdHJpbmcgc2VhcmNoIGZvciBub24tZGV0ZXJtaW5pc3RpYwogY29s
bGF0aW9ucwoKRHVlIHRvIGFuIG9mZi1ieS1vbmUgZXJyb3IsIHRoZSBjb2RlIGZhaWxlZCB0byBm
aW5kIG1hdGNoZXMgYXQgdGhlCmVuZCBvZiB0aGUgaGF5c3RhY2suCgpBdXRob3I6IExhdXJlbnog
QWxiZSA8bGF1cmVuei5hbGJlQGN5YmVydGVjLmF0PgpSZXBvcnRlZC1CeTogQWRhbSBXYXJsYW5k
IDxhZGFtLndhcmxhbmRAaW5mb3IuY29tPgpSZXZpZXdlZC1CeTogSGVpa2tpIExpbm5ha2FuZ2Fz
IDxobGlubmFrYUBpa2kuZmk+ClJldmlld2VkLUJ5OiBUb20gTGFuZSA8dGdsQHNzcy5wZ2gucGEu
dXM+CkRpc2N1c3Npb246IGh0dHBzOi8vcG9zdGdyLmVzL20vMTkzNDEtMWQ5YTIyOTE1ZWRmZWM1
OCU0MHBvc3RncmVzcWwub3JnCi0tLQogc3JjL2JhY2tlbmQvdXRpbHMvYWR0L3ZhcmxlbmEuYyAg
ICAgICAgICAgICAgICB8IDkgKysrKysrKystCiBzcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL2Nv
bGxhdGUuaWN1LnV0Zjgub3V0IHwgNyArKysrKysrCiBzcmMvdGVzdC9yZWdyZXNzL3NxbC9jb2xs
YXRlLmljdS51dGY4LnNxbCAgICAgIHwgMyArKysKIDMgZmlsZXMgY2hhbmdlZCwgMTggaW5zZXJ0
aW9ucygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3V0aWxzL2Fk
dC92YXJsZW5hLmMgYi9zcmMvYmFja2VuZC91dGlscy9hZHQvdmFybGVuYS5jCmluZGV4IDM4OTQ0
NTdhYjQwLi45OWQxYmYwOWYyMCAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvdXRpbHMvYWR0L3Zh
cmxlbmEuYworKysgYi9zcmMvYmFja2VuZC91dGlscy9hZHQvdmFybGVuYS5jCkBAIC0xMTM2LDYg
KzExMzYsOCBAQCB0ZXh0X3Bvc2l0aW9uX25leHRfaW50ZXJuYWwoY2hhciAqc3RhcnRfcHRyLCBU
ZXh0UG9zaXRpb25TdGF0ZSAqc3RhdGUpCiAJCWhwdHIgPSBzdGFydF9wdHI7CiAJCXdoaWxlICho
cHRyIDwgaGF5c3RhY2tfZW5kKQogCQl7CisJCQljb25zdCBjaGFyICp0ZXN0X2VuZDsKKwogCQkJ
LyoKIAkJCSAqIEZpcnN0IGNoZWNrIHRoZSBjb21tb24gY2FzZSB0aGF0IHRoZXJlIGlzIGEgbWF0
Y2ggaW4gdGhlCiAJCQkgKiBoYXlzdGFjayBvZiBleGFjdGx5IHRoZSBsZW5ndGggb2YgdGhlIG5l
ZWRsZS4KQEAgLTExNDksNyArMTE1MSw4IEBAIHRleHRfcG9zaXRpb25fbmV4dF9pbnRlcm5hbChj
aGFyICpzdGFydF9wdHIsIFRleHRQb3NpdGlvblN0YXRlICpzdGF0ZSkKIAkJCSAqIEVsc2UgY2hl
Y2sgaWYgYW55IG9mIHRoZSBwb3NzaWJsZSBzdWJzdHJpbmdzIHN0YXJ0aW5nIGF0IGhwdHIKIAkJ
CSAqIGFyZSBlcXVhbCB0byB0aGUgbmVlZGxlLgogCQkJICovCi0JCQlmb3IgKGNvbnN0IGNoYXIg
KnRlc3RfZW5kID0gaHB0cjsgdGVzdF9lbmQgPCBoYXlzdGFja19lbmQ7IHRlc3RfZW5kICs9IHBn
X21ibGVuKHRlc3RfZW5kKSkKKwkJCXRlc3RfZW5kID0gaHB0cjsKKwkJCXdoaWxlICh0ZXN0X2Vu
ZCA8PSBoYXlzdGFja19lbmQpCiAJCQl7CiAJCQkJaWYgKHBnX3N0cm5jb2xsKGhwdHIsICh0ZXN0
X2VuZCAtIGhwdHIpLCBuZWVkbGUsIG5lZWRsZV9sZW4sIHN0YXRlLT5sb2NhbGUpID09IDApCiAJ
CQkJewpAQCAtMTE1OCw2ICsxMTYxLDEwIEBAIHRleHRfcG9zaXRpb25fbmV4dF9pbnRlcm5hbChj
aGFyICpzdGFydF9wdHIsIFRleHRQb3NpdGlvblN0YXRlICpzdGF0ZSkKIAkJCQkJaWYgKCFzdGF0
ZS0+Z3JlZWR5KQogCQkJCQkJYnJlYWs7CiAJCQkJfQorCisJCQkJaWYgKHRlc3RfZW5kID09IGhh
eXN0YWNrX2VuZCkKKwkJCQkJYnJlYWs7CisJCQkJdGVzdF9lbmQgKz0gcGdfbWJsZW4odGVzdF9l
bmQpOwogCQkJfQogCQkJaWYgKHJlc3VsdF9ocHRyKQogCQkJCWJyZWFrOwpkaWZmIC0tZ2l0IGEv
c3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9jb2xsYXRlLmljdS51dGY4Lm91dCBiL3NyYy90ZXN0
L3JlZ3Jlc3MvZXhwZWN0ZWQvY29sbGF0ZS5pY3UudXRmOC5vdXQKaW5kZXggYjg1NzlhMWVmYzYu
LjBhMTRjMWQ5M2ZmIDEwMDY0NAotLS0gYS9zcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL2NvbGxh
dGUuaWN1LnV0Zjgub3V0CisrKyBiL3NyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQvY29sbGF0ZS5p
Y3UudXRmOC5vdXQKQEAgLTE0ODQsNiArMTQ4NCwxMyBAQCBTRUxFQ1QgYXJyYXlfc29ydCgne2Es
Qn0nOjp0ZXh0W10gQ09MTEFURSAiQyIpOwogIHtCLGF9CiAoMSByb3cpCiAKKy0tIHRlc3QgcmVw
bGFjZSgpIGF0IHRoZSBlbmQgb2YgdGhlIHN0cmluZyAoYnVnICMxOTM0MSkKK1NFTEVDVCByZXBs
YWNlKCd0ZXN0WCcgQ09MTEFURSBjYXNlX2luc2Vuc2l0aXZlLCAneCcgQ09MTEFURSBjYXNlX2lu
c2Vuc2l0aXZlLCAnZXMnKTsKKyByZXBsYWNlIAorLS0tLS0tLS0tCisgdGVzdGVzCisoMSByb3cp
CisKIC0tIHRlc3QgbGFuZ3VhZ2UgdGFncwogQ1JFQVRFIENPTExBVElPTiBsdF9pbnNlbnNpdGl2
ZSAocHJvdmlkZXIgPSBpY3UsIGxvY2FsZSA9ICdlbi11LWtzLWxldmVsMScsIGRldGVybWluaXN0
aWMgPSBmYWxzZSk7CiBTRUxFQ1QgJ2FCY0QnIENPTExBVEUgbHRfaW5zZW5zaXRpdmUgPSAnQWJD
ZCcgQ09MTEFURSBsdF9pbnNlbnNpdGl2ZTsKZGlmZiAtLWdpdCBhL3NyYy90ZXN0L3JlZ3Jlc3Mv
c3FsL2NvbGxhdGUuaWN1LnV0Zjguc3FsIGIvc3JjL3Rlc3QvcmVncmVzcy9zcWwvY29sbGF0ZS5p
Y3UudXRmOC5zcWwKaW5kZXggNmY1YWJhYzBkYzAuLjVlM2ZlZjliMzgxIDEwMDY0NAotLS0gYS9z
cmMvdGVzdC9yZWdyZXNzL3NxbC9jb2xsYXRlLmljdS51dGY4LnNxbAorKysgYi9zcmMvdGVzdC9y
ZWdyZXNzL3NxbC9jb2xsYXRlLmljdS51dGY4LnNxbApAQCAtNTY4LDYgKzU2OCw5IEBAIFNFTEVD
VCAnYWJjJyA8PSAnQUJDJyBDT0xMQVRFIGNhc2VfaW5zZW5zaXRpdmUsICdhYmMnID49ICdBQkMn
IENPTExBVEUgY2FzZV9pbnNlCiBTRUxFQ1QgYXJyYXlfc29ydCgne2EsQn0nOjp0ZXh0W10gQ09M
TEFURSBjYXNlX2luc2Vuc2l0aXZlKTsKIFNFTEVDVCBhcnJheV9zb3J0KCd7YSxCfSc6OnRleHRb
XSBDT0xMQVRFICJDIik7CiAKKy0tIHRlc3QgcmVwbGFjZSgpIGF0IHRoZSBlbmQgb2YgdGhlIHN0
cmluZyAoYnVnICMxOTM0MSkKK1NFTEVDVCByZXBsYWNlKCd0ZXN0WCcgQ09MTEFURSBjYXNlX2lu
c2Vuc2l0aXZlLCAneCcgQ09MTEFURSBjYXNlX2luc2Vuc2l0aXZlLCAnZXMnKTsKKwogLS0gdGVz
dCBsYW5ndWFnZSB0YWdzCiBDUkVBVEUgQ09MTEFUSU9OIGx0X2luc2Vuc2l0aXZlIChwcm92aWRl
ciA9IGljdSwgbG9jYWxlID0gJ2VuLXUta3MtbGV2ZWwxJywgZGV0ZXJtaW5pc3RpYyA9IGZhbHNl
KTsKIFNFTEVDVCAnYUJjRCcgQ09MTEFURSBsdF9pbnNlbnNpdGl2ZSA9ICdBYkNkJyBDT0xMQVRF
IGx0X2luc2Vuc2l0aXZlOwotLSAKMi41Mi4wCgo=


--=-E8vSkERV/qAhr9y48Fjx--