Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xDfLY-00000000djf-2EWp for pgsql-general@arkaria.postgresql.org; Mon, 05 Oct 2026 09:51:00 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xDfLW-00000009BtI-1m3J for pgsql-general@arkaria.postgresql.org; Mon, 05 Oct 2026 09:50:58 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xDfLW-00000009BtA-0OEF for pgsql-general@lists.postgresql.org; Mon, 05 Oct 2026 09:50:58 +0000 Received: from mail-ej2-x0f.google.com ([2a00:1450:4864:34::f]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1xDfLU-00000000RzR-0KAl for pgsql-general@lists.postgresql.org; Mon, 05 Oct 2026 09:50:56 +0000 Received: by mail-ej2-x0f.google.com with SMTP id a640c23a62f3a-c264788fd58so253759466b.1 for ; Mon, 05 Oct 2026 02:50:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1791193854; x=1791798654; darn=lists.postgresql.org; h=to:date:message-id:subject:mime-version:content-transfer-encoding :content-type:from:from:to:cc:subject:date:message-id:reply-to :content-type; bh=JOvLdyVeHwUWGNrCtvaTu0304M5oAhfUwcSqN44yaUw=; b=aJtiIEgBSSuHD0A+2FPoVTiInv11RSyqU3tmEANuJ5Bq2x2howhH+8/oymyiukyB+l Q5QAlYvyMUdAuSqSY3PcK8hE7bjsHEFVcXOsw95wxZMqgokLT5g98wlJT+X1JfVLDSCu pt7P2Ysawv0Emauf9ZCBbVUrP0ZzP4xMvEPAO4lsjV9jmzMr3h2qBMF8A+yUZCrnV1xB 9wertZk7fVHOIXt+LOxtUZk23Zpw//1suc3HisOQj5BYe7BIU3PpeCYH+VvrIl5jPN3f Dr2+iC+bPMsqnP8b9TX2I5UQ7f0qR6hmPk4TGtvFtwl+3DK3Y9aOj6OKmrnMJpifufh8 SBNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791193854; x=1791798654; h=to:date:message-id:subject:mime-version:content-transfer-encoding :content-type:from:x-gm-gg:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to:content-type; bh=JOvLdyVeHwUWGNrCtvaTu0304M5oAhfUwcSqN44yaUw=; b=Hjj4eIYKtv7U787aOKuSQdM/cWtygdR4+VRe/Cyu2GIBDKFPqNB8xt4XZGKdpZAP2+ JLK1YLIFseSppx9LqMfMywL5fY3fgZLrVz0i/9sSrN+PpDrXhyo3Wbp91aWsC1iN0C6X c6Oy5FtWX9l6ZA4hNgS0UW8p+qSo5wSyEL5cu9pIEU5/qL3KVl1NCkE2/0O9grdWSb3a 5We0iCIgxys4Iu3NUsR03dHqaXy9OuI61XfJ/ofIUPjd3ReCUkpjMvET8RGTssAreHQI QmYOGYFl4CEE/aZu2kG9sL4S68SJhtCY2F0GOzzuVb1dAPiPdQr5Azk+aNIXpz4Td0Tn xm+w== X-Gm-Message-State: AFuF++lyh9TAi93X4DkLfLQCUdfr6UNoEE1cmZrzDqJRAeDpf9h3128I I0RKyUvn0+grEZte/SaaXfi2UnKdRgo4YR3GTB3H7V0ANLki+Px3MgbVpyG5cg== X-Gm-Gg: AYBFou3ja3vBPMk/yTAUt6FADBtpqdpkYcAGJxTKdfPt7NFHZbfA2dyDsXw2ShTRXx0 Ar7C73FXdBWNMokqpqzARAs5GZ+QvH2g1WGOizfRJ7wf9yV+b/0PULhrnpuwVf6MsiOUUAB/4G+ Ls39s6sw89qasFZiWAXffyI+9Vax8sOGLZyoWnSI2FsLRnLg3PwUrvLi9Ny4lWdoIs6bZaa1Xbk HnoD1AR9m3WgRb6E4j/GvuVlC1odGRZbZrPh5dw8VDrlAtn3pYyOjIs4yYglyYaIopveCGn5O3Z gD443UeBuFAT3orfycgFaDnqHvHS4bIZk9lieF9r4He8cDjVTp8K5yXwgueJaOftbT2vIQeaj2x iQUFQUgSYWscwO7HC5GC2qkpuQ+kVyj2jIn/lUakW2bl4PrDDBLRB+alXIgeV1ZrsbFbMaQKx7r T7+HpoDTnye3hQR/bsHtMOIckzn8piUPliiTz68ZzDFB+1cpRPw47eZUSIiSTrdD4b8jJNz21Uz ps2og/DRLWcy0hX9VeNqt70gyAV+kcaUdZD/l73IVG8ipOcqtBlSmOa0IiDb4JFVbUG/wyJM8B+ tjul X-Received: by 2002:a17:906:1014:10b0:c2e:63ba:1f90 with SMTP id a640c23a62f3a-c2e63ba3e10mr467841266b.48.1791193853951; Mon, 05 Oct 2026 02:50:53 -0700 (PDT) Received: from smtpclient.apple (84-104-229-211.cable.dynamic.v4.ziggo.nl. [84.104.229.211]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-6afb0227a56sm3124422a12.34.2026.10.05.02.50.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Oct 2026 02:50:52 -0700 (PDT) From: Thomas de Zeeuw Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.700.51.1.1\)) Subject: Parsing of hex encoding strings Message-Id: Date: Mon, 5 Oct 2026 11:50:42 +0200 To: pgsql-general@lists.postgresql.org X-Mailer: Apple Mail (2.3864.700.51.1.1) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hello, I=E2=80=99m working on a system where I need to search for hex encoded = strings, for which I=E2=80=99m using full text search with the = =E2=80=9Csimple=E2=80=9D directory. This is working fine for the most = part, but I=E2=80=99ve run into an issue which may or may not be = considered a bug. When creating the ts vector and queries I want to achieve that the = entire hex encoded string is seen as a single word. For a string such as = =E2=80=9Cfe82fbb28=E2=80=9D this works using the =E2=80=9Csimple=E2=80=9D = directory. However a string such as =E2=80=9C7e82fbb28=E2=80=9D is = parsed as two words: =E2=80=9C7e82=E2=80=9D as scientific notation and = =E2=80=9Cfbb28" as numword. I=E2=80=99ve put the output of ts_debug at = the bottom of this email for convince. I would have expected that the entire string would be considered as one = token considering it doesn=E2=80=99t have any whitespace. And since that = token isn=E2=80=99t a valid scientific number as whole (though it can = be, just not the examples I=E2=80=99ve shared above) I would expected it = be considered a single word (numword). Reading the URL example from = chapter 12.5 = (https://www.postgresql.org/docs/current/textsearch-parsers.html) maybe = it could have to two tokens, one for the scientific number (=E2=80=9C7e82=E2= =80=9D) and another numword token for the entire string (7e82fbb28=E2=80=9D= ), this would also work for my use case. Finally, my question: is this considered behaviour a bug or working as = intended? Output of ts_debug: =E2=80=94 Wanted result. SELECT * FROM ts_debug('fe82fbb28'); = +---------+--------------------------+-----------+--------------+---------= ---+-------------+ | alias | description | token | dictionaries | = dictionary | lexemes | = +---------+--------------------------+-----------+--------------+---------= ---+-------------+ | numword | Word, letters and digits | fe82fbb28 | {simple} | simple = | {fe82fbb28} | = +---------+--------------------------+-----------+--------------+---------= ---+-------------+ =E2=80=94 Unwanted scientific notation. SELECT * FROM ts_debug('7e82fbb28'); = +---------+--------------------------+-------+--------------+------------+= ---------+ | alias | description | token | dictionaries | dictionary = | lexemes | = +---------+--------------------------+-------+--------------+------------+= ---------+ | sfloat | Scientific notation | 7e82 | {simple} | simple = | {7e82} | | numword | Word, letters and digits | fbb28 | {simple} | simple = | {fbb28} | = +---------+--------------------------+-------+--------------+------------+= ---------+ With kind regards, Thomas de Zeeuw