MIME-Version: 1.0
References: <19528-7290dd7e6f7dcc22@postgresql.org>
In-Reply-To: <19528-7290dd7e6f7dcc22@postgresql.org>
From: Fujii Masao <masao.fujii@gmail.com>
Date: Sun, 21 Jun 2026 00:18:19 +0900
Message-ID: 
 <CAHGQGwEGpf3eWyLMqQcxP11NyhvBNCFzmZP1EXM5F1Ca4NaLMQ@mail.gmail.com>
Subject: Re: BUG #19528: Assert failure in generate_normalized_query() via
 Squashed Array Literals
To: 3020001251@tju.edu.cn, pgsql-bugs@lists.postgresql.org
Content-Type: multipart/mixed; boundary="00000000000025e25f0654b0ec18"
Archived-At: 
 <https://www.postgresql.org/message-id/CAHGQGwEGpf3eWyLMqQcxP11NyhvBNCFzmZP1EXM5F1Ca4NaLMQ%40mail.gmail.com>
Precedence: bulk

--00000000000025e25f0654b0ec18
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, Jun 20, 2026 at 7:53=E2=80=AFPM PG Bug reporting form
<noreply@postgresql.org> wrote:
> `" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
> plus `" /*, ... */"` (11 bytes) =3D 13 bytes to replace a 1-byte constant=
, a
> net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes =
per
> squashed location. With N squashed array elements the buffer overflows by=
 2N
> bytes.

This analysis doesn't seem correct to me.

For "ARRAY[1,2]", the recorded location covers the entire list text,
"1,2" (3 bytes), which is replaced with "$n /*, ... */". For a
single-digit placeholder, "$1 /*, ... */" is 13 bytes, so replacing
"1,2" results in a net growth of 10 bytes.

That still fits within the existing budget. The budget is first
exceeded when the placeholder number reaches two digits:
"$10 /*, ... */" is 14 bytes, giving a net growth of 11 bytes.
Therefore, the smallest reproducer is 10 squashed "ARRAY[1,2]"
entries, where the shortfall seems only 1 byte overall, not 2 bytes per
squashed location.


> ### Fix
>
> Account for the extra 11 bytes appended when a squashed suffix is written=
.
> One correct approach:
>
> ```c
> norm_query_buflen =3D query_len + jstate->clocations_count *
>     (10 + (jstate->has_squashed_lists ? 11 : 0));
> ```
>
> A simpler conservative fix that covers the worst case (`"$2147483648 /*, =
...
> */"` =3D 22 bytes replacing a 1-byte constant, net 21 bytes):
>
> ```c
> norm_query_buflen =3D query_len + jstate->clocations_count * 22;
> ```

How about keeping the current "clocations_count * 10" budget for
ordinary replacements, counting squashed entries separately, and
adding only the extra space they may need, for example:

    clocations_count * 10 + squashed_count * 9

The existing "+10" already covers ordinary placeholders. For squashed
entries, the worst-case additional overhead is 9 bytes, from replacing
the shortest possible source text "1,2" (3 bytes) with the longest
possible placeholder "$2147483647 /*, ... */" (22 bytes), for a total
growth of 19 bytes. Attached patch implements this approach.

Thoughts?

Regards,

--=20
Fujii Masao

--00000000000025e25f0654b0ec18
Content-Type: application/octet-stream;
	name="v1-0001-pg_stat_statements-add-room-for-squashed-placehol.patch"
Content-Disposition: attachment;
	filename="v1-0001-pg_stat_statements-add-room-for-squashed-placehol.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mqmi5i020>
X-Attachment-Id: f_mqmi5i020

RnJvbSAwYjIyZDcxMzBkN2YxMzE1YWEzZDkxNjUyMGYyZGQ0YWEwY2ZhOTFiIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBGdWppaSBNYXNhbyA8ZnVqaWlAcG9zdGdyZXNxbC5vcmc+CkRh
dGU6IFNhdCwgMjAgSnVuIDIwMjYgMjM6MTE6NDEgKzA5MDAKU3ViamVjdDogW1BBVENIIHYxXSBw
Z19zdGF0X3N0YXRlbWVudHM6IGFkZCByb29tIGZvciBzcXVhc2hlZCBwbGFjZWhvbGRlcnMKCi0t
LQogLi4uL3BnX3N0YXRfc3RhdGVtZW50cy9leHBlY3RlZC9zcXVhc2hpbmcub3V0IHwgMjQgKysr
KysrKysrKysrKysrKysrKwogLi4uL3BnX3N0YXRfc3RhdGVtZW50cy9wZ19zdGF0X3N0YXRlbWVu
dHMuYyAgIHwgMTcgKysrKysrKysrKysrLQogY29udHJpYi9wZ19zdGF0X3N0YXRlbWVudHMvc3Fs
L3NxdWFzaGluZy5zcWwgIHwgIDcgKysrKysrCiAzIGZpbGVzIGNoYW5nZWQsIDQ3IGluc2VydGlv
bnMoKyksIDEgZGVsZXRpb24oLSkKCmRpZmYgLS1naXQgYS9jb250cmliL3BnX3N0YXRfc3RhdGVt
ZW50cy9leHBlY3RlZC9zcXVhc2hpbmcub3V0IGIvY29udHJpYi9wZ19zdGF0X3N0YXRlbWVudHMv
ZXhwZWN0ZWQvc3F1YXNoaW5nLm91dAppbmRleCA4NDM4MjM1YTJjZS4uMDU3ZjU2YzQ3NDEgMTAw
NjQ0Ci0tLSBhL2NvbnRyaWIvcGdfc3RhdF9zdGF0ZW1lbnRzL2V4cGVjdGVkL3NxdWFzaGluZy5v
dXQKKysrIGIvY29udHJpYi9wZ19zdGF0X3N0YXRlbWVudHMvZXhwZWN0ZWQvc3F1YXNoaW5nLm91
dApAQCAtNzksNiArNzksMzAgQEAgU0VMRUNUIHF1ZXJ5LCBjYWxscyBGUk9NIHBnX3N0YXRfc3Rh
dGVtZW50cyBPUkRFUiBCWSBxdWVyeSBDT0xMQVRFICJDIjsKICBTRUxFQ1QgcGdfc3RhdF9zdGF0
ZW1lbnRzX3Jlc2V0KCkgSVMgTk9UIE5VTEwgQVMgdCAgICB8ICAgICAxCiAoMyByb3dzKQogCist
LSBzbWFsbCBhcnJheSBsaXRlcmFscyBzdGlsbCBmaXQgb25jZSBzcXVhc2hlZCBwbGFjZWhvbGRl
cnMgcmVhY2ggMiBkaWdpdHMKK1NFTEVDVCBwZ19zdGF0X3N0YXRlbWVudHNfcmVzZXQoKSBJUyBO
T1QgTlVMTCBBUyB0OworIHQgCistLS0KKyB0CisoMSByb3cpCisKK1NFTEVDVCBBUlJBWVsxLDJd
IEFTIGExLCBBUlJBWVsxLDJdIEFTIGEyLCBBUlJBWVsxLDJdIEFTIGEzLCBBUlJBWVsxLDJdIEFT
IGE0LAorICAgICAgIEFSUkFZWzEsMl0gQVMgYTUsIEFSUkFZWzEsMl0gQVMgYTYsIEFSUkFZWzEs
Ml0gQVMgYTcsIEFSUkFZWzEsMl0gQVMgYTgsCisgICAgICAgQVJSQVlbMSwyXSBBUyBhOSwgQVJS
QVlbMSwyXSBBUyBhMTA7CisgIGExICAgfCAgYTIgICB8ICBhMyAgIHwgIGE0ICAgfCAgYTUgICB8
ICBhNiAgIHwgIGE3ICAgfCAgYTggICB8ICBhOSAgIHwgIGExMCAgCistLS0tLS0tKy0tLS0tLS0r
LS0tLS0tLSstLS0tLS0tKy0tLS0tLS0rLS0tLS0tLSstLS0tLS0tKy0tLS0tLS0rLS0tLS0tLSst
LS0tLS0tCisgezEsMn0gfCB7MSwyfSB8IHsxLDJ9IHwgezEsMn0gfCB7MSwyfSB8IHsxLDJ9IHwg
ezEsMn0gfCB7MSwyfSB8IHsxLDJ9IHwgezEsMn0KKygxIHJvdykKKworU0VMRUNUIHF1ZXJ5LCBj
YWxscyBGUk9NIHBnX3N0YXRfc3RhdGVtZW50cyBPUkRFUiBCWSBxdWVyeSBDT0xMQVRFICJDIjsK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHF1ZXJ5ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHwgY2FsbHMgCistLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0tLQorIFNFTEVDVCBBUlJBWVskMSAvKiwgLi4u
ICovXSBBUyBhMSwgQVJSQVlbJDIgLyosIC4uLiAqL10gQVMgYTIsIEFSUkFZWyQzIC8qLCAuLi4g
Ki9dIEFTIGEzLCBBUlJBWVskNCAvKiwgLi4uICovXSBBUyBhNCwrfCAgICAgMQorICAgICAgICBB
UlJBWVskNSAvKiwgLi4uICovXSBBUyBhNSwgQVJSQVlbJDYgLyosIC4uLiAqL10gQVMgYTYsIEFS
UkFZWyQ3IC8qLCAuLi4gKi9dIEFTIGE3LCBBUlJBWVskOCAvKiwgLi4uICovXSBBUyBhOCwrfCAK
KyAgICAgICAgQVJSQVlbJDkgLyosIC4uLiAqL10gQVMgYTksIEFSUkFZWyQxMCAvKiwgLi4uICov
XSBBUyBhMTAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHwgCisgU0VMRUNUIHBnX3N0YXRfc3RhdGVtZW50c19yZXNldCgpIElTIE5PVCBOVUxM
IEFTIHQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB8ICAgICAxCisoMiByb3dzKQorCiAtLSBidWlsdC1pbiBmdW5jdGlv
bnMgd2lsbCBiZSBzcXVhc2hlZAogLS0gdGhlIElOIGFuZCBBUlJBWSBmb3JtcyBvZiB0aGlzIHN0
YXRlbWVudCB3aWxsIGhhdmUgdGhlIHNhbWUgcXVlcnlJZAogU0VMRUNUIHBnX3N0YXRfc3RhdGVt
ZW50c19yZXNldCgpIElTIE5PVCBOVUxMIEFTIHQ7CmRpZmYgLS1naXQgYS9jb250cmliL3BnX3N0
YXRfc3RhdGVtZW50cy9wZ19zdGF0X3N0YXRlbWVudHMuYyBiL2NvbnRyaWIvcGdfc3RhdF9zdGF0
ZW1lbnRzL3BnX3N0YXRfc3RhdGVtZW50cy5jCmluZGV4IDkyMzE1NjI3OTE2Li5iZDc5YTdkMTQ3
MCAxMDA2NDQKLS0tIGEvY29udHJpYi9wZ19zdGF0X3N0YXRlbWVudHMvcGdfc3RhdF9zdGF0ZW1l
bnRzLmMKKysrIGIvY29udHJpYi9wZ19zdGF0X3N0YXRlbWVudHMvcGdfc3RhdF9zdGF0ZW1lbnRz
LmMKQEAgLTI4MjIsNiArMjgyMiw3IEBAIGdlbmVyYXRlX25vcm1hbGl6ZWRfcXVlcnkoY29uc3Qg
SnVtYmxlU3RhdGUgKmpzdGF0ZSwgY29uc3QgY2hhciAqcXVlcnksCiAJCQkJbGFzdF9vZmYgPSAw
LAkvKiBPZmZzZXQgZnJvbSBzdGFydCBmb3IgcHJldmlvdXMgdG9rICovCiAJCQkJbGFzdF90b2tf
bGVuID0gMDsJLyogTGVuZ3RoIChpbiBieXRlcykgb2YgdGhhdCB0b2sgKi8KIAlpbnQJCQludW1f
Y29uc3RhbnRzX3JlcGxhY2VkID0gMDsKKwlpbnQJCQlzcXVhc2hlZF9jb3VudCA9IDA7CiAJTG9j
YXRpb25MZW4gKmxvY3MgPSBOVUxMOwogCiAJLyoKQEAgLTI4MzcsOCArMjgzOCwyMiBAQCBnZW5l
cmF0ZV9ub3JtYWxpemVkX3F1ZXJ5KGNvbnN0IEp1bWJsZVN0YXRlICpqc3RhdGUsIGNvbnN0IGNo
YXIgKnF1ZXJ5LAogCSAqIGNlcnRhaW5seSBpc24ndCBtb3JlIHRoYW4gMTEgYnl0ZXMsIGV2ZW4g
aWYgbiByZWFjaGVzIElOVF9NQVguICBXZQogCSAqIGNvdWxkIHJlZmluZSB0aGF0IGxpbWl0IGJh
c2VkIG9uIHRoZSBtYXggdmFsdWUgb2YgbiBmb3IgdGhlIGN1cnJlbnQKIAkgKiBxdWVyeSwgYnV0
IGl0IGhhcmRseSBzZWVtcyB3b3J0aCBhbnkgZXh0cmEgZWZmb3J0IHRvIGRvIHNvLgorCSAqCisJ
ICogU3F1YXNoZWQgbGlzdHMgbmVlZCBhIGxpdHRsZSBtb3JlIHJvb20uIFRoZSBzaG9ydGVzdCBz
cXVhc2hlZCBzb3VyY2UKKwkgKiB0ZXh0IGlzICIxLDIiICh0aHJlZSBieXRlcyksIHdoaWxlIHRo
ZSBsb25nZXN0IHNxdWFzaGVkIHBsYWNlaG9sZGVyIGlzCisJICogMjIgYnl0ZXMgbG9uZyAoYW4g
SU5UX01BWCBwbGFjZWhvbGRlciBwbHVzIHRoZSBzcXVhc2ggbWFya2VyKSwgc28gZWFjaAorCSAq
IHNxdWFzaGVkIGxpc3QgY2FuIG5lZWQgdXAgdG8gOSBieXRlcyBtb3JlIHRoYW4gdGhlIGJhc2Ug
YWxsb3dhbmNlIG9mIDEwCisJICogYnl0ZXMgcGVyIGVudHJ5LgogCSAqLwotCW5vcm1fcXVlcnlf
YnVmbGVuID0gcXVlcnlfbGVuICsganN0YXRlLT5jbG9jYXRpb25zX2NvdW50ICogMTA7CisJZm9y
IChpbnQgaSA9IDA7IGkgPCBqc3RhdGUtPmNsb2NhdGlvbnNfY291bnQ7IGkrKykKKwl7CisJCS8q
IElnbm9yZSBkdXBsaWNhdGUgbG9jYXRpb25zIHRoYXQgQ29tcHV0ZUNvbnN0YW50TGVuZ3Rocygp
IGRpc2FibGVkICovCisJCWlmIChsb2NzW2ldLnNxdWFzaGVkICYmIGxvY3NbaV0ubGVuZ3RoID49
IDApCisJCQlzcXVhc2hlZF9jb3VudCsrOworCX0KKworCW5vcm1fcXVlcnlfYnVmbGVuID0gcXVl
cnlfbGVuICsganN0YXRlLT5jbG9jYXRpb25zX2NvdW50ICogMTAgKworCQlzcXVhc2hlZF9jb3Vu
dCAqIDk7CiAKIAkvKiBBbGxvY2F0ZSByZXN1bHQgYnVmZmVyICovCiAJbm9ybV9xdWVyeSA9IHBh
bGxvYyhub3JtX3F1ZXJ5X2J1ZmxlbiArIDEpOwpkaWZmIC0tZ2l0IGEvY29udHJpYi9wZ19zdGF0
X3N0YXRlbWVudHMvc3FsL3NxdWFzaGluZy5zcWwgYi9jb250cmliL3BnX3N0YXRfc3RhdGVtZW50
cy9zcWwvc3F1YXNoaW5nLnNxbAppbmRleCBmYzllNjU3Mzg3My4uMmJkMjZiMjRlZWMgMTAwNjQ0
Ci0tLSBhL2NvbnRyaWIvcGdfc3RhdF9zdGF0ZW1lbnRzL3NxbC9zcXVhc2hpbmcuc3FsCisrKyBi
L2NvbnRyaWIvcGdfc3RhdF9zdGF0ZW1lbnRzL3NxbC9zcXVhc2hpbmcuc3FsCkBAIC0yNCw2ICsy
NCwxMyBAQCBTRUxFQ1QgQVJSQVlbMSwgMiwgMywgNF07CiBTRUxFQ1QgQVJSQVlbMSwgMiwgMywg
NCwgNV07CiBTRUxFQ1QgcXVlcnksIGNhbGxzIEZST00gcGdfc3RhdF9zdGF0ZW1lbnRzIE9SREVS
IEJZIHF1ZXJ5IENPTExBVEUgIkMiOwogCistLSBzbWFsbCBhcnJheSBsaXRlcmFscyBzdGlsbCBm
aXQgb25jZSBzcXVhc2hlZCBwbGFjZWhvbGRlcnMgcmVhY2ggMiBkaWdpdHMKK1NFTEVDVCBwZ19z
dGF0X3N0YXRlbWVudHNfcmVzZXQoKSBJUyBOT1QgTlVMTCBBUyB0OworU0VMRUNUIEFSUkFZWzEs
Ml0gQVMgYTEsIEFSUkFZWzEsMl0gQVMgYTIsIEFSUkFZWzEsMl0gQVMgYTMsIEFSUkFZWzEsMl0g
QVMgYTQsCisgICAgICAgQVJSQVlbMSwyXSBBUyBhNSwgQVJSQVlbMSwyXSBBUyBhNiwgQVJSQVlb
MSwyXSBBUyBhNywgQVJSQVlbMSwyXSBBUyBhOCwKKyAgICAgICBBUlJBWVsxLDJdIEFTIGE5LCBB
UlJBWVsxLDJdIEFTIGExMDsKK1NFTEVDVCBxdWVyeSwgY2FsbHMgRlJPTSBwZ19zdGF0X3N0YXRl
bWVudHMgT1JERVIgQlkgcXVlcnkgQ09MTEFURSAiQyI7CisKIC0tIGJ1aWx0LWluIGZ1bmN0aW9u
cyB3aWxsIGJlIHNxdWFzaGVkCiAtLSB0aGUgSU4gYW5kIEFSUkFZIGZvcm1zIG9mIHRoaXMgc3Rh
dGVtZW50IHdpbGwgaGF2ZSB0aGUgc2FtZSBxdWVyeUlkCiBTRUxFQ1QgcGdfc3RhdF9zdGF0ZW1l
bnRzX3Jlc2V0KCkgSVMgTk9UIE5VTEwgQVMgdDsKLS0gCjIuNTMuMAoK
--00000000000025e25f0654b0ec18--