Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uIOuv-003qUe-1N for pgsql-bugs@arkaria.postgresql.org; Fri, 23 May 2025 09:42:17 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1uIOut-000LDR-VA for pgsql-bugs@arkaria.postgresql.org; Fri, 23 May 2025 09:42:15 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uIOut-000L9Z-H8 for pgsql-bugs@lists.postgresql.org; Fri, 23 May 2025 09:42:15 +0000 Received: from mail-ej1-x629.google.com ([2a00:1450:4864:20::629]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1uIOuq-000Ufz-2m for pgsql-bugs@lists.postgresql.org; Fri, 23 May 2025 09:42:14 +0000 Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-ad560321ed9so783161866b.1 for ; Fri, 23 May 2025 02:42:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec.at; s=google; t=1747993331; x=1748598131; darn=lists.postgresql.org; h=mime-version:user-agent:references:in-reply-to:date:cc:to:from :subject:message-id:from:to:cc:subject:date:message-id:reply-to; bh=TqHKlwVrbItwUTjr5RN5+IXYY6KL1nSvYXg4mS7c+Zc=; b=edi9MkyeHsIlidkCDbGRNXUhyzKpvy4iNO5oqll918ByNZktaZroRaOkhC2bC3H7av 7LN8c3eT27yPDeOTaWOqwKGXEr7XnDXmWAxKWc8FFgdQrARpDolnbcVZ5KtdXzMOy5JC 93pJ3AAxWfACl8cxzsFsulJGJzHIgbvgfqQRVB/Jmc1D4XpWHH8hgqkYZZCArU18vtIu vU1CixAq3l2GZ5V4azRFzUIMf6VVdHge6sR5Z8/DRiakQWKVoavKVYG6alMBDdG/fECF bd4h56iVgqUHu5ly/Q0y7p7VwhJBbNkSObseYCfQ+HCqXkGQ4ouWT+ZSvsoGI9uXSW1t dzXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747993331; x=1748598131; h=mime-version:user-agent:references:in-reply-to:date:cc:to:from :subject:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=TqHKlwVrbItwUTjr5RN5+IXYY6KL1nSvYXg4mS7c+Zc=; b=P5WCE4wBhOLAl3X1hv5UCwCVMvYsPB14CdmBTysL0H05M3XtkCfVADyG+f5zohyIiN oObaqnLMCwO5qUtFHBX9c9UyKh9opBEJKN5ZMwbfIl79YygdD7v09Wvq7oVEFzJgQabO lS5xlP8YYAl3PPMPO0Yajl6TyxKIgAu7XYtMunmRrN0NE2i0KDr4SUiYR/F/pvur2I0t YSuyTMVDfmtl+wxOHAsg6NN4daATAVdTSn+7ZHv+mvnmkJPHL4W48lC89bIGEEDpSPgN YSRFLr34Z+ziHBcLnL43hOM+Z8xVXue2Asv9WW5SZcS9kZ76mmdj1GEKMQm8cmIzBJrX vfFQ== X-Gm-Message-State: AOJu0YwqUtNdkzKZGIANlMKigSCed6Xue+A4AMQFkYCIpYf7n+NyQiAk VYz5OOvrxtw0WaPnRf7l8V9RHzBiUQsp5KhMQnfEnuJoWtbYw2M7h5z75ssqEL3YR6A= X-Gm-Gg: ASbGnctORtGe0KedXBN/gc++0pQ9/BWa8JaZ1yyxajLTIEnAZjZDnaaErAWie11eQOT uGze/q4MIMrRz7pu/cca0pcatKSXUcg+7FBj2xTCS4MQy4z6uca2B07apxFoK4IFs35uPaiUjbf fE1DpVRR5TPSZs2ouq5T6MGKaQhXEojJURzMS1XiiUDvQqt9DqdCqCmiGbV4e5FneutgXHFDaIo MbneJLL2BZvWbYkF/BYX2wpT81Lify0kY6oETGNpi0HlMSb+sntu6RNgNZI1Puc8X21Sjo4c+MY yFd/QLzNCEghmpBlqHIcTXlgs6p27CfbmRidKDNHYObUMnG4sNjKavRoSAl/uTyG1laQgDQbp// pPu7M X-Google-Smtp-Source: AGHT+IF8qkFjQy+lWoc/j0yzJmzuhzOw9iuu8TX3jcMrLr00lXDJMfIpg62PQMabT50Iv27x76xNbw== X-Received: by 2002:a17:907:db02:b0:ad5:27b6:7fd1 with SMTP id a640c23a62f3a-ad52d48da04mr2764821666b.17.1747993331429; Fri, 23 May 2025 02:42:11 -0700 (PDT) Received: from laurenz.albe-K4N0CV00F97414D ([2001:871:5e:6c9b:73f5:6897:6da1:73ea]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ad52d04b0a1sm1212852766b.12.2025.05.23.02.42.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 May 2025 02:42:10 -0700 (PDT) Message-ID: <5498224d1d5a6a3be7f29ae7725f76162d48372f.camel@cybertec.at> Subject: Re: SIMILAR TO expressions translate wildcards where they shouldn't From: Laurenz Albe To: Michael Paquier Cc: pgsql-bugs@lists.postgresql.org Date: Fri, 23 May 2025 11:42:10 +0200 In-Reply-To: References: <16ab039d1af455652bdf4173402ddda145f2c73b.camel@cybertec.at> Content-Type: multipart/mixed; boundary="=-0UzlyZjfHocepqysteBB" User-Agent: Evolution 3.56.1 (3.56.1-1.fc42) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --=-0UzlyZjfHocepqysteBB Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, 2025-05-23 at 12:22 +0900, Michael Paquier wrote: > Anyway, I don't think that the tests in the patch are complete. For > example, '[[^](]' is transformed into '^(?:[[^](])$' in the SIMILAR TO > conversion with the patch, and before the patch we get > '^(?:[[^](?:])$'. Note the translation of the last parenthesis '(' to > "(?:" when inside the character class, but your patch does not > document that. AFAIU, we should not convert the parenthesis '(' while > in a multi-level character class as the patch does, but we have no > tests for it and the patch does not document this part, either. >=20 > Could it be possible to split the single SIMILAR TO expression into > multiple smaller pieces for each character that should not be > converted while inside a character class? This is hard to parse as > written in your proposal of patch. I have changed the regression test like you suggest. I also improved the code by adding more comments. I renamed "incharclass" to "charclass_depth", which is more descriptive. Also, I had to work some more on handling carets: While the closing bracket is a regular character in []] and [^]], it is not in expressions like [^^]. Yours, Laurenz Albe --=-0UzlyZjfHocepqysteBB Content-Disposition: attachment; filename*0=v2-0001-Fix-SIMILAR-TO-regex-translation-for-character-cl.pat; filename*1=ch Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="v2-0001-Fix-SIMILAR-TO-regex-translation-for-character-cl.patch"; charset="UTF-8" RnJvbSBmMGM1NmYxZjk1YWIzNWRjYTNkN2ZmNjVmNzgzMjg5YTZiMjI1MTQ2IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBMYXVyZW56IEFsYmUgPGxhdXJlbnouYWxiZUBjeWJlcnRlYy5h dD4KRGF0ZTogRnJpLCAyMyBNYXkgMjAyNSAxMTozNTozMiArMDIwMApTdWJqZWN0OiBbUEFUQ0gg djJdIEZpeCBTSU1JTEFSIFRPIHJlZ2V4IHRyYW5zbGF0aW9uIGZvciBjaGFyYWN0ZXIgY2xhc3Nl cwoKVGhlIGNvZGUgdGhhdCB0cmFuc2xhdGVzIFNJTUlMQVIgVE8gcGF0dGVybiBtYXRjaGluZyBl eHByZXNzaW9ucwp0byByZWd1bGFyIGV4cHJlc3Npb25zIGRpZG4ndCBjb25zaWRlciB0aGF0IGJy YWNrZXRzIGNhbiBiZSBuZXN0ZWQsCmFzIGluIFtbOmFscGhhOl0lXSwgYW5kIHJlcGxhY2VkIHBs YWNlaG9sZGVycyBsaWtlIF8gYW5kICUgd2hlcmUKaXQgc2hvdWxkbid0LgoKRml4IGJ5IHRyYWNr aW5nIHRoZSBuZXN0aW5nIGxldmVsIG9mIGJyYWNrZXRzIHdoaWxlIGNvbnNpZGVyaW5nIHRoYXQK aW4gZXhwcmVzc2lvbnMgbGlrZSBbXV0gYW5kIFteXV0gdGhlIGZpcnN0IGNsb3NpbmcgYnJhY2tl dCBpcyBhCnJlZ3VsYXIgY2hhcmFjdGVyLgoKQXV0aG9yOiBMYXVyZW56IEFsYmUgPGxhdXJlbnou YWxiZUBjeWJlcnRlYy5hdD4KUmV2aWV3ZWQtYnk6IE1pY2hhZWwgUGFxdWllciA8bWljaGFlbEBw YXF1aWVyLnh5ej4KRGlzY3Vzc2lvbjogaHR0cHM6Ly9wb3N0Z3IuZXMvbS8xNmFiMDM5ZDFhZjQ1 NTY1MmJkZjQxNzM0MDJkZGRhMTQ1ZjJjNzNiLmNhbWVsQGN5YmVydGVjLmF0CkJhY2twYXRjaC10 aHJvdWdoOiAxMwotLS0KIHNyYy9iYWNrZW5kL3V0aWxzL2FkdC9yZWdleHAuYyAgICAgICAgfCAz NiArKysrKysrKysrKysrKysrKysrKysrLS0tLS0KIHNyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQv c3RyaW5ncy5vdXQgfCAxMiArKysrKysrKysKIHNyYy90ZXN0L3JlZ3Jlc3Mvc3FsL3N0cmluZ3Mu c3FsICAgICAgfCAgMyArKysKIDMgZmlsZXMgY2hhbmdlZCwgNDUgaW5zZXJ0aW9ucygrKSwgNiBk ZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC91dGlscy9hZHQvcmVnZXhwLmMg Yi9zcmMvYmFja2VuZC91dGlscy9hZHQvcmVnZXhwLmMKaW5kZXggZWRlZTFmNzg4MGIuLjlkNmY3 Njc1YzY0IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC91dGlscy9hZHQvcmVnZXhwLmMKKysrIGIv c3JjL2JhY2tlbmQvdXRpbHMvYWR0L3JlZ2V4cC5jCkBAIC03NzMsNyArNzczLDggQEAgc2ltaWxh cl9lc2NhcGVfaW50ZXJuYWwodGV4dCAqcGF0X3RleHQsIHRleHQgKmVzY190ZXh0KQogCWludAkJ CXBsZW4sCiAJCQkJZWxlbjsKIAlib29sCQlhZnRlcmVzY2FwZSA9IGZhbHNlOwotCWJvb2wJCWlu Y2hhcmNsYXNzID0gZmFsc2U7CisJaW50CQkJY2hhcmNsYXNzX2RlcHRoID0gMDsJLyogc3FhcmUg YnJhY2tldCBuZXN0aW5nIGxldmVsICovCisJaW50CQkJY2hhcmNsYXNzX3N0YXJ0ID0gMDsKIAlp bnQJCQlucXVvdGVzID0gMDsKIAogCXAgPSBWQVJEQVRBX0FOWShwYXRfdGV4dCk7CkBAIC05MDQs NyArOTA1LDcgQEAgc2ltaWxhcl9lc2NhcGVfaW50ZXJuYWwodGV4dCAqcGF0X3RleHQsIHRleHQg KmVzY190ZXh0KQogCQkvKiBmYXN0IHBhdGggKi8KIAkJaWYgKGFmdGVyZXNjYXBlKQogCQl7Ci0J CQlpZiAocGNoYXIgPT0gJyInICYmICFpbmNoYXJjbGFzcykJLyogZXNjYXBlLWRvdWJsZS1xdW90 ZT8gKi8KKwkJCWlmIChwY2hhciA9PSAnIicgJiYgY2hhcmNsYXNzX2RlcHRoIDwgMSkJLyogZXNj YXBlLWRvdWJsZS1xdW90ZT8gKi8KIAkJCXsKIAkJCQkvKiBlbWl0IGFwcHJvcHJpYXRlIHBhcnQg c2VwYXJhdG9yLCBwZXIgbm90ZXMgYWJvdmUgKi8KIAkJCQlpZiAobnF1b3RlcyA9PSAwKQpAQCAt OTUzLDE4ICs5NTQsNDEgQEAgc2ltaWxhcl9lc2NhcGVfaW50ZXJuYWwodGV4dCAqcGF0X3RleHQs IHRleHQgKmVzY190ZXh0KQogCQkJLyogU1FMIGVzY2FwZSBjaGFyYWN0ZXI7IGRvIG5vdCBzZW5k IHRvIG91dHB1dCAqLwogCQkJYWZ0ZXJlc2NhcGUgPSB0cnVlOwogCQl9Ci0JCWVsc2UgaWYgKGlu Y2hhcmNsYXNzKQorCQllbHNlIGlmIChjaGFyY2xhc3NfZGVwdGggPiAwKQogCQl7CiAJCQlpZiAo cGNoYXIgPT0gJ1xcJykKIAkJCQkqcisrID0gJ1xcJzsKIAkJCSpyKysgPSBwY2hhcjsKLQkJCWlm IChwY2hhciA9PSAnXScpCi0JCQkJaW5jaGFyY2xhc3MgPSBmYWxzZTsKKworCQkJLyoKKwkJCSAq IElnbm9yZSBhIGNsb3NpbmcgYnJhY2tldCBhdCB0aGUgc3RhcnQgb2YgYSBjaGFyYWN0ZXIgY2xh c3MuCisJCQkgKiBTdWNoIGEgYnJhY2tldCBpcyB0YWtlbiBsaXRlcmFsbHkgcmF0aGVyIHRoYW4g Y2xvc2luZyB0aGUKKwkJCSAqIGNsYXNzLiAiY2hhcmNsYXNzX3N0YXJ0IiBpcyAxIHJpZ2h0IGF0 IHRoZSBiZWdpbm5pbmcgb2YgYSBjbGFzcworCQkJICogYW5kIDIgYWZ0ZXIgYW4gaW5pdGlhbCBj YXJldC4KKwkJCSAqLworCQkJaWYgKHBjaGFyID09ICdbJykKKwkJCQljaGFyY2xhc3NfZGVwdGgr KzsKKwkJCWVsc2UgaWYgKHBjaGFyID09ICddJyAmJiBjaGFyY2xhc3Nfc3RhcnQgPiAyKQorCQkJ CWNoYXJjbGFzc19kZXB0aC0tOworCisJCQkvKgorCQkJICogSWYgdGhlcmUgaXMgYSBjYXJldCBy aWdodCBhZnRlciB0aGUgb3BlbmluZyBicmFja2V0LCBpdCBuZWdhdGVzCisJCQkgKiB0aGUgY2hh cmFjdGVyIGNsYXNzLCBidXQgYSBmb2xsb3dpbmcgY2xvc2luZyBicmFja2V0IHNob3VsZAorCQkJ ICogc3RpbGwgYmUgdHJlYXRlZCBhcyBhIG5vcm1hbCBjaGFyYWN0ZXIuICBIb3dldmVyLCB0aGF0 IGhvbGRzCisJCQkgKiBvbmx5IGZvciB0aGUgZmlyc3QgY2FyZXQsIHNvIG9ubHkgdGhlIHZhbHVl cyAxIGFuZCAyIG1lYW4gdGhhdAorCQkJICogY2xvc2luZyBicmFja2V0cyBzaG91bGQgYmUgdGFr ZW4gbGl0ZXJhbGx5LgorCQkJICovCisJCQlpZiAocGNoYXIgPT0gJ14nKQorCQkJCWNoYXJjbGFz c19zdGFydCsrOworCQkJZWxzZQorCQkJCWNoYXJjbGFzc19zdGFydCA9IDM7CS8qIGRlZmluaXRl bHkgcGFzdCB0aGUgc3RhcnQgKi8KIAkJfQogCQllbHNlIGlmIChwY2hhciA9PSAnWycpCiAJCXsK KwkJCS8qIHN0YXJ0IG9mIGEgY2hhcmFjdGVyIGNsYXNzICovCiAJCQkqcisrID0gcGNoYXI7Ci0J CQlpbmNoYXJjbGFzcyA9IHRydWU7CisJCQljaGFyY2xhc3NfZGVwdGgrKzsKKwkJCWNoYXJjbGFz c19zdGFydCA9IDE7CiAJCX0KIAkJZWxzZSBpZiAocGNoYXIgPT0gJyUnKQogCQl7CmRpZmYgLS1n aXQgYS9zcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL3N0cmluZ3Mub3V0IGIvc3JjL3Rlc3QvcmVn cmVzcy9leHBlY3RlZC9zdHJpbmdzLm91dAppbmRleCAxNzRmMGE2ODMzMS4uY2UyMzA5NjFiNWMg MTAwNjQ0Ci0tLSBhL3NyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQvc3RyaW5ncy5vdXQKKysrIGIv c3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9zdHJpbmdzLm91dApAQCAtNjE0LDYgKzYxNCwxOCBA QCBTRUxFQ1QgJ2FiY2RlZmcnIFNJTUlMQVIgVE8gJ19iY2QlJyBFU0NBUEUgTlVMTCBBUyBudWxs OwogU0VMRUNUICdhYmNkZWZnJyBTSU1JTEFSIFRPICdfYmNkIyUnIEVTQ0FQRSAnIyMnIEFTIGVy cm9yOwogRVJST1I6ICBpbnZhbGlkIGVzY2FwZSBzdHJpbmcKIEhJTlQ6ICBFc2NhcGUgc3RyaW5n IG11c3QgYmUgZW1wdHkgb3Igb25lIGNoYXJhY3Rlci4KKy0tICIuIiwgIl8iLCAiJSIsICIkIiwg IigiIGFuZCAiXiIgc2hvdWxkIGJlIGxlZnQgYWxvbmUgaW4gY2hhcmFjdGVyIGNsYXNzZXMKK0VY UExBSU4gKFZFUkJPU0UsIENPU1RTIE9GRikKK1NFTEVDVCAoU0VMRUNUICcnKSBTSU1JTEFSIFRP ICcuWy5bOmFsbnVtOl1fXV9bXSVdJVteXSRdJFteXl1eWyhdKHApJzsKKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBRVUVSWSBQTEFOICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAKKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0K KyBSZXN1bHQKKyAgIE91dHB1dDogKChJbml0UGxhbiAxKS5jb2wxIH4gJ14oPzpcLlsuWzphbG51 bTpdX10uW10lXS4qW15dJF1cJFteXl1cXlsoXSg/OnApKSQnOjp0ZXh0KQorICAgSW5pdFBsYW4g MQorICAgICAtPiAgUmVzdWx0CisgICAgICAgICAgIE91dHB1dDogJyc6OnRleHQKKyg1IHJvd3Mp CisKIC0tIFRlc3QgYmFja3NsYXNoIGVzY2FwZXMgaW4gcmVnZXhwX3JlcGxhY2UncyByZXBsYWNl bWVudCBzdHJpbmcKIFNFTEVDVCByZWdleHBfcmVwbGFjZSgnMTExMjIyMzMzMycsIEUnKFxcZHsz fSkoXFxkezN9KShcXGR7NH0pJywgRScoXFwxKSBcXDItXFwzJyk7CiAgcmVnZXhwX3JlcGxhY2Ug CmRpZmYgLS1naXQgYS9zcmMvdGVzdC9yZWdyZXNzL3NxbC9zdHJpbmdzLnNxbCBiL3NyYy90ZXN0 L3JlZ3Jlc3Mvc3FsL3N0cmluZ3Muc3FsCmluZGV4IGY3YjMyNWJhYWRmLi45NGI5YWE0OGU5MCAx MDA2NDQKLS0tIGEvc3JjL3Rlc3QvcmVncmVzcy9zcWwvc3RyaW5ncy5zcWwKKysrIGIvc3JjL3Rl c3QvcmVncmVzcy9zcWwvc3RyaW5ncy5zcWwKQEAgLTE5Niw2ICsxOTYsOSBAQCBTRUxFQ1QgJ2Fi Y2RcZWZnJyBTSU1JTEFSIFRPICdfYmNkXCUnIEVTQ0FQRSAnJyBBUyB0cnVlOwogLS0gdGhlc2Ug YmVoYXZpb3JzIGFyZSBwZXIgc3BlYywgdGhvdWdoOgogU0VMRUNUICdhYmNkZWZnJyBTSU1JTEFS IFRPICdfYmNkJScgRVNDQVBFIE5VTEwgQVMgbnVsbDsKIFNFTEVDVCAnYWJjZGVmZycgU0lNSUxB UiBUTyAnX2JjZCMlJyBFU0NBUEUgJyMjJyBBUyBlcnJvcjsKKy0tICIuIiwgIl8iLCAiJSIsICIk IiwgIigiIGFuZCAiXiIgc2hvdWxkIGJlIGxlZnQgYWxvbmUgaW4gY2hhcmFjdGVyIGNsYXNzZXMK K0VYUExBSU4gKFZFUkJPU0UsIENPU1RTIE9GRikKK1NFTEVDVCAoU0VMRUNUICcnKSBTSU1JTEFS IFRPICcuWy5bOmFsbnVtOl1fXV9bXSVdJVteXSRdJFteXl1eWyhdKHApJzsKIAogLS0gVGVzdCBi YWNrc2xhc2ggZXNjYXBlcyBpbiByZWdleHBfcmVwbGFjZSdzIHJlcGxhY2VtZW50IHN0cmluZwog U0VMRUNUIHJlZ2V4cF9yZXBsYWNlKCcxMTEyMjIzMzMzJywgRScoXFxkezN9KShcXGR7M30pKFxc ZHs0fSknLCBFJyhcXDEpIFxcMi1cXDMnKTsKLS0gCjIuNDkuMAoK --=-0UzlyZjfHocepqysteBB--