pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Thomas de Zeeuw <thomasdezeeuw@gmail.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Parsing of hex encoding strings
Date: Mon, 05 Oct 2026 12:35:17 +0200
Message-ID: <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at> (raw)
In-Reply-To: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com>
References: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com>

On Mon, 2026-10-05 at 11:50 +0200, Thomas de Zeeuw wrote:
> I’m working on a system where I need to search for hex encoded strings, for
> which I’m using full text search with the “simple” directory. This is working
> fine for the most part, but I’ve run into an issue which may or may not be
> considered a bug.
> 
> When creating the ts vector and queries I want to achieve that the entire hex
> encoded string is seen as a single word. For a string such as “fe82fbb28”
> this works using the “simple” directory. However a string such as “7e82fbb28”
> is parsed as two words: “7e82” as scientific notation and “fbb28" as numword.
> I’ve put the output of ts_debug at the bottom of this email for convince.
> 
> I would have expected that the entire string would be considered as one token
> considering it doesn’t have any whitespace. And since that token isn’t a valid
> scientific number as whole (though it can be, just not the examples I’ve
> shared above) I would expected it be considered a single word (numword).
> Reading the URL example from chapter 12.5
> (https://www.postgresql.org/docs/current/textsearch-parsers.html) maybe it
> could have to two tokens, one for the scientific number (“7e82”) and another
> numword token for the entire string (7e82fbb28”), this would also work for my
> use case.
> 
> Finally, my question: is this considered behaviour a bug or working as intended?

That is working as intended.  The problem is that you are trying to use full-text
search for something that isn't a text.  Storing strings in hex encoding was a
design error.  Not only does it keep you from using full-text search, but you
are also wasting storage space (unless TOAST compression takes care of that).

You could use substring search with pg_trgm, but it is not the same as
full-text search.

Yours,
Laurenz Albe





view thread (5+ messages)  latest in thread

Message-ID: <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at>
Permalink:  ../088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at/
Also on:    postgresql.org/message-id/088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: laurenz.albe@cybertec.at, thomasdezeeuw@gmail.com
  Subject: Re: Parsing of hex encoding strings
  In-Reply-To: <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at>
  References: <CC9C0072-98AF-4014-820C-F65BAF575346@gmail.com> <088bd47f443ee4b8f657e3dbfadf611a560aa95d.camel@cybertec.at>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox