From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Parsing of hex encoding strings
Date: Mon, 05 Oct 2026 12:35:17 +0200
Message-ID: <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at> (raw)
In-Reply-To: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com>
References: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com>
On Mon, 2026-10-05 at 11:50 +0200, Thomas de Zeeuw wrote:
> I’m working on a system where I need to search for hex encoded strings, for
> which I’m using full text search with the “simple” directory. This is working
> fine for the most part, but I’ve run into an issue which may or may not be
> considered a bug.
>
> When creating the ts vector and queries I want to achieve that the entire hex
> encoded string is seen as a single word. For a string such as “fe82fbb28”
> this works using the “simple” directory. However a string such as “7e82fbb28”
> is parsed as two words: “7e82” as scientific notation and “fbb28" as numword.
> I’ve put the output of ts_debug at the bottom of this email for convince.
>
> I would have expected that the entire string would be considered as one token
> considering it doesn’t have any whitespace. And since that token isn’t a valid
> scientific number as whole (though it can be, just not the examples I’ve
> shared above) I would expected it be considered a single word (numword).
> Reading the URL example from chapter 12.5
> (https://www.postgresql.org/docs/current/textsearch-parsers.html) maybe it
> could have to two tokens, one for the scientific number (“7e82”) and another
> numword token for the entire string (7e82fbb28”), this would also work for my
> use case.
>
> Finally, my question: is this considered behaviour a bug or working as intended?
That is working as intended. The problem is that you are trying to use full-text
search for something that isn't a text. Storing strings in hex encoding was a
design error. Not only does it keep you from using full-text search, but you
are also wasting storage space (unless TOAST compression takes care of that).
You could use substring search with pg_trgm, but it is not the same as
full-text search.
Yours,
Laurenz Albe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: laurenz.albe@cybertec.at, thomasdezeeuw@gmail.com
Subject: Re: Parsing of hex encoding strings
In-Reply-To: <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at>
References: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com> <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox