agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedReduce build times of pg_trgm GIN indexes
51+ messages / 10 participants
[nested] [flat]
* Reduce build times of pg_trgm GIN indexes
@ 2026-01-05 15:01 David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-01-05 15:01 UTC (permalink / raw)
To: pgsql-hackers
Hi hackers,
Attached is a series of patches which gradually reduce the time it takes
to create GIN indexes. Most of the gains come from optimizing the
trigram extraction code in pg_trgm. A few small optimizations apply to
any GIN index operator class.
The changes are motivated by the time it takes to create GIN indexes on
large production tables, especially, on columns with long strings. Even
with multiple parallel maintenance workers I've seen this taking hours.
For testing purposes I've used two different data sets:
1. The l_comment column of the TPC-H SF 10 lineitem table. l_comment
contains relatively short strings with a minimum of 10, a maximum of 43
and an average of ~27 characters.
2. The plots from a collection of movies from Wikipedia. The plots are
much longer than l_comment, with a minimum of 15, a maximum of 36,773
and an average of ~2,165 characters. The CSV file can be downloaded here
[1].
Testing both cases is important because a big part of the trigram
extraction is spent on removing duplicates. The longer the string, the
more duplicates are usually encountered.
The script I used for testing is attached. I ran CREATE INDEX three
times and took the fastest run. I'm getting the following results on my
i9-13950HX dev laptop in release build:
Data set | Patched (ms) | Master (ms) | Speedup
--------------------|--------------|--------------|----------
movies(plot) | 3,409 | 10,311 | 3.02x
lineitem(l_comment) | 161,569 | 256,986 | 1.59x
The attached patches do the following:
- v1-0001-Inline-ginCompareAttEntries.patch: Inline
ginCompareAttEntries() which is very frequently called by the GIN code.
- v1-0002-Optimized-comparison-functions.patch: Use FunctionCallInvoke()
instead of FunctionCall2Coll(). This saves a bunch of per-comparison
setup code, such as calling InitFunctionCallInfoData().
- v1-0003-Use-sort_template.h.patch: Use sort_template.h instead of
qsort(), to inline calls to the sort comparator. This is an interim step
that is further improved on by patch 0006.
- v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
ginExtractEntries() deduplicates and sorts the entries returned from the
extract value function. In case of pg_trgm, that is completely redundant
because the trigrams are already deduplicated and sorted. The current
version of this patch is just to demonstrate the potential. We need to
think about what we want here. Ideally, we would require the extraction
function to provide the entries deduplicated and sorted. Alternatively,
we could indicate to ginExtractEntries() if the entries are already
deduplicated and sorted. If we don't want to alter the signature of the
extract value function, we could e.g. use the MSB of the nentries argument.
- v1-0005-Make-btint4cmp-branchless.patch: Removes branches from
btint4cmp(), which is heavily called from the GIN code. This might as
well have benefit in other parts of the code base.
v1-0006-Use-radix-sort.patch: Replace the sort_template.h-based qsort()
with radix sort. For the purpose of demonstrating the possible gains,
I've only replaced the signed variant for now. I've also tried using
simplehash.h for deduplicating followed by a sort_template.h-based sort.
But that was slower.
v1-0007-Faster-qunique-comparator.patch: qunique() doesn't require a
full sort comparator (-1 = less, 0 = equal, 1 = greater) but only a
equal/unequal comparator (e.g. 0 = equal and 1 = unequal). The same
optimization can be done in plenty of other places in our code base.
Likely, in most of them the gains are insignificant.
v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch: Typically lots
of text is actually ASCII. Hence, we provide a fast path for this case
which is exercised if the MSB of the current character is unset.
With above changes, the majority of the runtime is now spent on
inserting the trigrams into the GIN index via ginInsertBAEntry(). The
code in master uses a red-black for further deduplication and sorting.
Traversing the red-black tree and updating it is pretty slow. I haven't
looked through all the code yet, but it seems to me that we would be
better off replacing the red-black tree with a sort and/or a hash map.
But I'll leave this as future work for now.
[1]
https://github.com/kiq005/movie-recommendation/raw/refs/heads/master/src/dataset/wiki_movie_plots_de...
--
David Geier
Attachments:
[text/x-patch] v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch (4.0K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/2-v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch)
download | inline diff:
From fc7bf3bd9c1bdf1336c233a796079556bb1dbf9d Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Fri, 14 Nov 2025 11:37:40 +0100
Subject: [PATCH v1 8/8] Add ASCII fastpath to generate_trgm_only()
---
contrib/pg_trgm/trgm_op.c | 124 ++++++++++++++++++++------------------
1 file changed, 65 insertions(+), 59 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index e07c553b1bd..42343bd1442 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,32 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-/*
- * Finds first word in string, returns pointer to the word,
- * endword points to the character after word
- */
-static char *
-find_word(char *str, int lenstr, char **endword, int *charlen)
-{
- char *beginword = str;
-
- while (beginword - str < lenstr && !ISWORDCHR(beginword))
- beginword += pg_mblen(beginword);
-
- if (beginword - str >= lenstr)
- return NULL;
-
- *endword = beginword;
- *charlen = 0;
- while (*endword - str < lenstr && ISWORDCHR(*endword))
- {
- *endword += pg_mblen(*endword);
- (*charlen)++;
- }
-
- return beginword;
-}
-
/*
* Reduce a trigram (three possibly multi-byte characters) to a trgm,
* which is always exactly three bytes. If we have three single-byte
@@ -337,58 +311,90 @@ make_trigrams(trgm *tptr, char *str, int bytelen, int charlen)
static int
generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
{
- trgm *tptr;
- char *buf;
- int charlen,
- bytelen;
- char *bword,
- *eword;
+ trgm *tptr = trg;
+ char *buf;
if (slen + LPADDING + RPADDING < 3 || slen == 0)
return 0;
- tptr = trg;
+ buf = palloc_array(char, slen * pg_database_encoding_max_length() + 4 + 1);
+ memset(buf, ' ', LPADDING);
- /* Allocate a buffer for case-folded, blank-padded words */
- buf = (char *) palloc(slen * pg_database_encoding_max_length() + 4);
-
- if (LPADDING > 0)
+ for (int i = 0; i < slen; )
{
- *buf = ' ';
- if (LPADDING > 1)
- *(buf + 1) = ' ';
- }
+ int num_bytes = LPADDING;
+ int num_chars = LPADDING;
+ char *word;
- eword = str;
- while ((bword = find_word(eword, slen - (eword - str), &eword, &charlen)) != NULL)
- {
+ /* Extract next word */
+ while (i < slen)
+ {
+ if ((str[i] & 0x80) == 0) /* Fast path for ASCII-only */
+ {
+ if (isalnum(str[i]))
+ {
#ifdef IGNORECASE
- bword = str_tolower(bword, eword - bword, DEFAULT_COLLATION_OID);
- bytelen = strlen(bword);
+ buf[num_bytes++] = pg_ascii_tolower(str[i++]);
#else
- bytelen = eword - bword;
+ buf[num_bytes++] = str[i++];
#endif
+ }
+ else
+ {
+ i++;
+ break;
+ }
+ }
+ else
+ {
+ const int mblen = pg_mblen(str + i);
+ Assert(mblen >= 2); /* Otherwise, it would be ASCII */
+
+ if (ISWORDCHR(str + i))
+ {
+ memcpy(buf + num_bytes, str + i, mblen);
+ num_bytes += mblen;
+ i += mblen;
+ }
+ else
+ {
+ i += mblen;
+ break;
+ }
+ }
+
+ num_chars++;
+ }
- memcpy(buf + LPADDING, bword, bytelen);
+ if (num_chars > LPADDING)
+ {
+ memset(buf + num_bytes, ' ', RPADDING);
+ num_bytes += RPADDING;
+ num_chars += RPADDING;
+ word = buf;
#ifdef IGNORECASE
- pfree(bword);
+ if (num_chars != num_bytes)
+ {
+ word = str_tolower(buf, num_bytes, DEFAULT_COLLATION_OID);
+ num_bytes = strlen(word); /* String can get shorter from lower-casing */
+ }
#endif
- buf[LPADDING + bytelen] = ' ';
- buf[LPADDING + bytelen + 1] = ' ';
+ if (bounds)
+ bounds[tptr - trg] |= TRGM_BOUND_LEFT;
+
+ tptr = make_trigrams(tptr, word, num_bytes, num_chars);
+
+ if (bounds)
+ bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
- /* Calculate trigrams marking their bounds if needed */
- if (bounds)
- bounds[tptr - trg] |= TRGM_BOUND_LEFT;
- tptr = make_trigrams(tptr, buf, bytelen + LPADDING + RPADDING,
- charlen + LPADDING + RPADDING);
- if (bounds)
- bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
+ if (word != buf)
+ pfree(word);
+ }
}
pfree(buf);
-
return tptr - trg;
}
--
2.51.0
[text/x-patch] v1-0007-Faster-qunique-comparator.patch (1.9K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/3-v1-0007-Faster-qunique-comparator.patch)
download | inline diff:
From 4d101725a6101d723858d5b7561ff1cef123f671 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 12 Nov 2025 14:27:13 +0100
Subject: [PATCH v1 7/8] Faster qunique() comparator
---
contrib/pg_trgm/trgm_op.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 25ee049f352..e07c553b1bd 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -139,6 +139,14 @@ CMPTRGM_UNSIGNED(const void *a, const void *b)
: CMPPCHAR_UNS(a, b, 2));
}
+static inline int
+CMPTRGM_EQ(const void *a, const void *b)
+{
+ char *aa = (char *)a;
+ char *bb = (char *)b;
+ return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
+}
+
/*
* This gets called on the first call. It replaces the function pointer so
* that subsequent calls are routed directly to the chosen implementation.
@@ -482,15 +490,11 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1038,15 +1042,11 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v1-0006-Use-radix-sort.patch (2.9K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/4-v1-0006-Use-radix-sort.patch)
download | inline diff:
From 6148b4bbc702eb4ce89994f3aa4790def50ed0c5 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v1 6/8] Use radix sort
---
contrib/pg_trgm/trgm_op.c | 64 +++++++++++++++++++++++++++++++++------
1 file changed, 54 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 875065a4670..25ee049f352 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -155,14 +155,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
}
/* Define our specialized sort function name */
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
@@ -392,6 +384,58 @@ generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
return tptr - trg;
}
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ Assert(to == buffer);
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
+
/*
* Guard against possible overflow in the palloc requests below. (We
* don't worry about the additive constants, since palloc can detect
@@ -439,7 +483,7 @@ generate_trgm(char *str, int slen)
{
if (GetDefaultCharSignedness())
{
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
}
else
@@ -995,7 +1039,7 @@ generate_wildcard_trgm(const char *str, int slen)
{
if (GetDefaultCharSignedness())
{
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
}
else
--
2.51.0
[text/x-patch] v1-0005-Make-btint4cmp-branchless.patch (1.0K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/5-v1-0005-Make-btint4cmp-branchless.patch)
download | inline diff:
From 915bd326de5ebde4149093d52fff9c710e557de2 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v1 5/8] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index bffc4b7709c..5ae27c22621 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -60,6 +60,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -202,12 +203,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
[text/x-patch] v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch (1.8K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/6-v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch)
download | inline diff:
From e91198730d10d301857b1ef38670f506fc0749ca Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 14:40:37 +0100
Subject: [PATCH v1 4/8] Avoid dedup and sort in ginExtractEntries
---
src/backend/access/gin/ginutil.c | 21 +++++++++------------
1 file changed, 9 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index f4139effd6e..9187264dbdc 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -498,13 +498,6 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
return entries;
}
- /*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
- */
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
-
/*
* If there's more than one key, sort and unique-ify.
*
@@ -512,8 +505,8 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
- {
+ if (*nentries > 1 && nullFlags != NULL)
+ {
keyEntryData *keydata;
cmpEntriesArg arg;
@@ -564,9 +557,13 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
/*
* Create GinNullCategory representation from nullFlags.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY is 0");
+ *categories = palloc0_array(GinNullCategory, *nentries);
+
+ if (nullFlags != NULL)
+ for (i = 0; i < *nentries; i++)
+ if (nullFlags[i])
+ (*categories)[i] = GIN_CAT_NULL_KEY;
return entries;
}
--
2.51.0
[text/x-patch] v1-0003-Use-sort_template.h.patch (2.6K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/7-v1-0003-Use-sort_template.h.patch)
download | inline diff:
From 5a5b3b955fffac4baf2ea1367914ce87a6f8bf5a Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:11 +0100
Subject: [PATCH v1 3/8] Use sort_template.h
---
contrib/pg_trgm/trgm_op.c | 49 ++++++++++++++++++++++++++++++---------
1 file changed, 38 insertions(+), 11 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 81182a15e07..875065a4670 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -154,6 +154,23 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
+/* Define our specialized sort function name */
+#define ST_SORT trigram_qsort_signed
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
+#define ST_SORT trigram_qsort_unsigned
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
/*
* Deprecated function.
* Use "pg_trgm.similarity_threshold" GUC variable instead of this function.
@@ -209,12 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-static int
-comp_trgm(const void *a, const void *b)
-{
- return CMPTRGM(a, b);
-}
-
/*
* Finds first word in string, returns pointer to the word,
* endword points to the character after word
@@ -426,12 +437,20 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
-
+
return trg;
}
@@ -974,8 +993,16 @@ generate_wildcard_trgm(const char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v1-0002-Optimized-comparison-functions.patch (4.0K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/8-v1-0002-Optimized-comparison-functions.patch)
download | inline diff:
From eb7fe3763b31d30d48ebbd4934085fe70315249a Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:01 +0100
Subject: [PATCH v1 2/8] Optimized comparison functions
---
src/backend/access/gin/ginutil.c | 19 ++++++++++++-------
src/include/access/gin_private.h | 19 +++++++++++++++----
2 files changed, 27 insertions(+), 11 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d205093e21d..f4139effd6e 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -114,6 +114,7 @@ initGinState(GinState *state, Relation index)
for (i = 0; i < origTupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(origTupdesc, i);
+ FunctionCallInfoBaseData *fci = &state->compareFnCallInfo[i].fcinfo;
if (state->oneCol)
state->tupdesc[i] = state->origTupdesc;
@@ -222,6 +223,10 @@ initGinState(GinState *state, Relation index)
state->supportCollation[i] = index->rd_indcollation[i];
else
state->supportCollation[i] = DEFAULT_COLLATION_OID;
+
+ InitFunctionCallInfoData(*fci, &state->compareFn[i], 2, state->supportCollation[i], NULL, NULL);
+ fci->args[0].isnull = false;
+ fci->args[1].isnull = false;
}
}
@@ -402,8 +407,7 @@ typedef struct
typedef struct
{
- FmgrInfo *cmpDatumFunc;
- Oid collation;
+ FunctionCallInfoBaseData *cmpFuncInfo;
bool haveDups;
} cmpEntriesArg;
@@ -425,9 +429,11 @@ cmpEntries(const void *a, const void *b, void *arg)
else if (bb->isnull)
res = -1; /* not-NULL "<" NULL */
else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ {
+ data->cmpFuncInfo->args[0].value = aa->datum;
+ data->cmpFuncInfo->args[1].value = bb->datum;
+ res = DatumGetInt32(FunctionCallInvoke(data->cmpFuncInfo));
+ }
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -518,8 +524,7 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
keydata[i].isnull = nullFlags[i];
}
- arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
- arg.collation = ginstate->supportCollation[attnum - 1];
+ arg.cmpFuncInfo = &ginstate->compareFnCallInfo[attnum - 1].fcinfo;
arg.haveDups = false;
qsort_arg(keydata, *nentries, sizeof(keyEntryData),
cmpEntries, &arg);
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index e155045ce8a..7cf19c8a5dc 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -51,6 +51,11 @@ typedef struct GinOptions
#define GIN_SHARE BUFFER_LOCK_SHARE
#define GIN_EXCLUSIVE BUFFER_LOCK_EXCLUSIVE
+typedef union CompareFuncCallInfoData
+{
+ FunctionCallInfoBaseData fcinfo;
+ char fcinfo_data[SizeForFunctionCallInfo(2)];
+} CompareFuncCallInfoData;
/*
* GinState: working data structure describing the index being worked on
@@ -77,6 +82,10 @@ typedef struct GinState
/*
* Per-index-column opclass support functions
*/
+
+
+ CompareFuncCallInfoData compareFnCallInfo[INDEX_MAX_KEYS];
+
FmgrInfo compareFn[INDEX_MAX_KEYS];
FmgrInfo extractValueFn[INDEX_MAX_KEYS];
FmgrInfo extractQueryFn[INDEX_MAX_KEYS];
@@ -504,6 +513,8 @@ ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
Datum a, GinNullCategory categorya,
Datum b, GinNullCategory categoryb)
{
+ FunctionCallInfoBaseData *fci;
+
/* if not of same null category, sort by that first */
if (categorya != categoryb)
return (categorya < categoryb) ? -1 : 1;
@@ -512,10 +523,10 @@ ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
if (categorya != GIN_CAT_NORM_KEY)
return 0;
- /* both not null, so safe to call the compareFn */
- return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
- ginstate->supportCollation[attnum - 1],
- a, b));
+ fci = &ginstate->compareFnCallInfo[attnum - 1].fcinfo;
+ fci->args[0].value = a;
+ fci->args[1].value = b;
+ return DatumGetInt32(FunctionCallInvoke(fci));
}
/*
--
2.51.0
[text/x-patch] v1-0001-Inline-ginCompareAttEntries.patch (4.0K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/9-v1-0001-Inline-ginCompareAttEntries.patch)
download | inline diff:
From a484e7c69bec62474b041eb4e533601e4883dab3 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Thu, 6 Nov 2025 09:42:27 +0100
Subject: [PATCH v1 1/8] Inline ginCompareAttEntries
---
src/backend/access/gin/ginutil.c | 38 ---------------------------
src/include/access/gin_private.h | 44 +++++++++++++++++++++++++++-----
2 files changed, 38 insertions(+), 44 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index a546cac18d3..d205093e21d 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -387,44 +387,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Compare two keys of the same index column
- */
-int
-ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
- Datum a, GinNullCategory categorya,
- Datum b, GinNullCategory categoryb)
-{
- /* if not of same null category, sort by that first */
- if (categorya != categoryb)
- return (categorya < categoryb) ? -1 : 1;
-
- /* all null items in same category are equal */
- if (categorya != GIN_CAT_NORM_KEY)
- return 0;
-
- /* both not null, so safe to call the compareFn */
- return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
- ginstate->supportCollation[attnum - 1],
- a, b));
-}
-
-/*
- * Compare two keys of possibly different index columns
- */
-int
-ginCompareAttEntries(GinState *ginstate,
- OffsetNumber attnuma, Datum a, GinNullCategory categorya,
- OffsetNumber attnumb, Datum b, GinNullCategory categoryb)
-{
- /* attribute number is the first sort key */
- if (attnuma != attnumb)
- return (attnuma < attnumb) ? -1 : 1;
-
- return ginCompareEntries(ginstate, attnuma, a, categorya, b, categoryb);
-}
-
-
/*
* Support for sorting key datums in ginExtractEntries
*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index b33f7cec5b4..e155045ce8a 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -97,12 +97,6 @@ extern Buffer GinNewBuffer(Relation index);
extern void GinInitBuffer(Buffer b, uint32 f);
extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
-extern int ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
- Datum a, GinNullCategory categorya,
- Datum b, GinNullCategory categoryb);
-extern int ginCompareAttEntries(GinState *ginstate,
- OffsetNumber attnuma, Datum a, GinNullCategory categorya,
- OffsetNumber attnumb, Datum b, GinNullCategory categoryb);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
int32 *nentries, GinNullCategory **categories);
@@ -502,6 +496,44 @@ ginCompareItemPointers(ItemPointer a, ItemPointer b)
return pg_cmp_u64(ia, ib);
}
+/*
+ * Compare two keys of the same index column
+ */
+static inline int
+ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
+ Datum a, GinNullCategory categorya,
+ Datum b, GinNullCategory categoryb)
+{
+ /* if not of same null category, sort by that first */
+ if (categorya != categoryb)
+ return (categorya < categoryb) ? -1 : 1;
+
+ /* all null items in same category are equal */
+ if (categorya != GIN_CAT_NORM_KEY)
+ return 0;
+
+ /* both not null, so safe to call the compareFn */
+ return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
+ ginstate->supportCollation[attnum - 1],
+ a, b));
+}
+
+/*
+ * Compare two keys of possibly different index columns
+ */
+static inline int
+ginCompareAttEntries(GinState *ginstate,
+ OffsetNumber attnuma, Datum a, GinNullCategory categorya,
+ OffsetNumber attnumb, Datum b, GinNullCategory categoryb)
+{
+ /* attribute number is the first sort key */
+ if (attnuma != attnumb)
+ return (attnuma < attnumb) ? -1 : 1;
+
+ return ginCompareEntries(ginstate, attnuma, a, categorya, b, categoryb);
+
+}
+
extern int ginTraverseLock(Buffer buffer, bool searchMode);
#endif /* GIN_PRIVATE_H */
--
2.51.0
[application/sql] test_gin_optimizations.sql (1.2K, ../../5d366878-2007-4d31-861e-19294b7a583b@gmail.com/10-test_gin_optimizations.sql)
download
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-06 17:00 Heikki Linnakangas <hlinnaka@iki.fi>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 2 replies; 51+ messages in thread
From: Heikki Linnakangas @ 2026-01-06 17:00 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; pgsql-hackers
On 05/01/2026 17:01, David Geier wrote:
> The script I used for testing is attached. I ran CREATE INDEX three
> times and took the fastest run. I'm getting the following results on my
> i9-13950HX dev laptop in release build:
>
> Data set | Patched (ms) | Master (ms) | Speedup
> --------------------|--------------|--------------|----------
> movies(plot) | 3,409 | 10,311 | 3.02x
> lineitem(l_comment) | 161,569 | 256,986 | 1.59x
>
Impressive speedup!
> The attached patches do the following:
>
> - v1-0001-Inline-ginCompareAttEntries.patch: Inline
> ginCompareAttEntries() which is very frequently called by the GIN code.
Looks good.
> - v1-0002-Optimized-comparison-functions.patch: Use FunctionCallInvoke()
> instead of FunctionCall2Coll(). This saves a bunch of per-comparison
> setup code, such as calling InitFunctionCallInfoData().
You lose the check for NULL result with this. That's probably still
worth checking.
> - v1-0003-Use-sort_template.h.patch: Use sort_template.h instead of
> qsort(), to inline calls to the sort comparator. This is an interim step
> that is further improved on by patch 0006.
ok
> - v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
> ginExtractEntries() deduplicates and sorts the entries returned from the
> extract value function. In case of pg_trgm, that is completely redundant
> because the trigrams are already deduplicated and sorted. The current
> version of this patch is just to demonstrate the potential. We need to
> think about what we want here. Ideally, we would require the extraction
> function to provide the entries deduplicated and sorted. Alternatively,
> we could indicate to ginExtractEntries() if the entries are already
> deduplicated and sorted. If we don't want to alter the signature of the
> extract value function, we could e.g. use the MSB of the nentries argument.
Yeah, this seems wrong as it is. You're assuming that if the extract
function returns nullFlags == NULL, the array is already sorted and deduped.
> - v1-0005-Make-btint4cmp-branchless.patch: Removes branches from
> btint4cmp(), which is heavily called from the GIN code. This might as
> well have benefit in other parts of the code base.
Seems reasonable.
> v1-0006-Use-radix-sort.patch: Replace the sort_template.h-based qsort()
> with radix sort. For the purpose of demonstrating the possible gains,
> I've only replaced the signed variant for now. I've also tried using
> simplehash.h for deduplicating followed by a sort_template.h-based sort.
> But that was slower.
Ok.
> v1-0007-Faster-qunique-comparator.patch: qunique() doesn't require a
> full sort comparator (-1 = less, 0 = equal, 1 = greater) but only a
> equal/unequal comparator (e.g. 0 = equal and 1 = unequal). The same
> optimization can be done in plenty of other places in our code base.
> Likely, in most of them the gains are insignificant.
Makes sense. I'm a little disappointed the compiler won't do that
optimization for us..
Perhaps we should introduce a new qunique_eq() function with a different
callback signature:
/* like qunique(), but the callback function returns true/false rather
than int */
static inline size_t
qunique_eq(void *array, size_t elements, size_t width,
bool (*equal) (const void *, const void *))
> v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch: Typically lots
> of text is actually ASCII. Hence, we provide a fast path for this case
> which is exercised if the MSB of the current character is unset.
This uses pg_ascii_tolower() when for ASCII characters when built with
the IGNORECASE. I don't think that's correct, if the proper collation
would do something more complicated for than what pg_ascii_tolower() does.
Did you measure how big is the impact from each individual patch?
Patches 1 and 2 seem pretty much ready to be committed, but I wonder if
they make any difference on their own.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-06 20:05 Kirill Reshke <reshkekirill@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 1 reply; 51+ messages in thread
From: Kirill Reshke @ 2026-01-06 20:05 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: David Geier <geidav.pg@gmail.com>; pgsql-hackers
On Tue, 6 Jan 2026 at 22:00, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> On 05/01/2026 17:01, David Geier wrote:
> > The script I used for testing is attached. I ran CREATE INDEX three
> > times and took the fastest run. I'm getting the following results on my
> > i9-13950HX dev laptop in release build:
> >
> > Data set | Patched (ms) | Master (ms) | Speedup
> > --------------------|--------------|--------------|----------
> > movies(plot) | 3,409 | 10,311 | 3.02x
> > lineitem(l_comment) | 161,569 | 256,986 | 1.59x
> >
>
> Impressive speedup!
>
> > The attached patches do the following:
> >
> > - v1-0001-Inline-ginCompareAttEntries.patch: Inline
> > ginCompareAttEntries() which is very frequently called by the GIN code.
>
> Looks good.
>
> > - v1-0002-Optimized-comparison-functions.patch: Use FunctionCallInvoke()
> > instead of FunctionCall2Coll(). This saves a bunch of per-comparison
> > setup code, such as calling InitFunctionCallInfoData().
>
> You lose the check for NULL result with this. That's probably still
> worth checking.
>
> > - v1-0003-Use-sort_template.h.patch: Use sort_template.h instead of
> > qsort(), to inline calls to the sort comparator. This is an interim step
> > that is further improved on by patch 0006.
>
> ok
>
> > - v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
> > ginExtractEntries() deduplicates and sorts the entries returned from the
> > extract value function. In case of pg_trgm, that is completely redundant
> > because the trigrams are already deduplicated and sorted. The current
> > version of this patch is just to demonstrate the potential. We need to
> > think about what we want here. Ideally, we would require the extraction
> > function to provide the entries deduplicated and sorted. Alternatively,
> > we could indicate to ginExtractEntries() if the entries are already
> > deduplicated and sorted. If we don't want to alter the signature of the
> > extract value function, we could e.g. use the MSB of the nentries argument.
>
> Yeah, this seems wrong as it is. You're assuming that if the extract
> function returns nullFlags == NULL, the array is already sorted and deduped.
>
> > - v1-0005-Make-btint4cmp-branchless.patch: Removes branches from
> > btint4cmp(), which is heavily called from the GIN code. This might as
> > well have benefit in other parts of the code base.
>
> Seems reasonable.
>
> > v1-0006-Use-radix-sort.patch: Replace the sort_template.h-based qsort()
> > with radix sort. For the purpose of demonstrating the possible gains,
> > I've only replaced the signed variant for now. I've also tried using
> > simplehash.h for deduplicating followed by a sort_template.h-based sort.
> > But that was slower.
>
> Ok.
>
> > v1-0007-Faster-qunique-comparator.patch: qunique() doesn't require a
> > full sort comparator (-1 = less, 0 = equal, 1 = greater) but only a
> > equal/unequal comparator (e.g. 0 = equal and 1 = unequal). The same
> > optimization can be done in plenty of other places in our code base.
> > Likely, in most of them the gains are insignificant.
>
> Makes sense. I'm a little disappointed the compiler won't do that
> optimization for us..
>
> Perhaps we should introduce a new qunique_eq() function with a different
> callback signature:
>
> /* like qunique(), but the callback function returns true/false rather
> than int */
> static inline size_t
> qunique_eq(void *array, size_t elements, size_t width,
> bool (*equal) (const void *, const void *))
>
> > v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch: Typically lots
> > of text is actually ASCII. Hence, we provide a fast path for this case
> > which is exercised if the MSB of the current character is unset.
>
> This uses pg_ascii_tolower() when for ASCII characters when built with
> the IGNORECASE. I don't think that's correct, if the proper collation
> would do something more complicated for than what pg_ascii_tolower() does.
>
> Did you measure how big is the impact from each individual patch?
> Patches 1 and 2 seem pretty much ready to be committed, but I wonder if
> they make any difference on their own.
>
> - Heikki
>
>
>
Hi!
patches 0003, 0004 & 0008 applies with whitespace errors.
reshke@yezzey-cbdb-bench:~/pgpure$ git am v1-0003-Use-sort_template.h.patch
Applying: Use sort_template.h
.git/rebase-apply/patch:66: trailing whitespace.
warning: 1 line adds whitespace errors.
reshke@yezzey-cbdb-bench:~/pgpure$ git am
v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
Applying: Avoid dedup and sort in ginExtractEntries
.git/rebase-apply/patch:30: trailing whitespace.
{
warning: 1 line adds whitespace errors.
reshke@yezzey-cbdb-bench:~/pgpure$ git am
v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch
Applying: Add ASCII fastpath to generate_trgm_only()
.git/rebase-apply/patch:101: trailing whitespace.
else
warning: 1 line adds whitespace errors.
I did run benchmarks of my VM using your data. v1-0001 solely improves
runtime by 4-5%. v2-0002 does not show runtime improvement for me.
With parallel GIN build, performance gains are 2-3 % for 2 workers and
not noticeable after that.
I will try to run some more benchmarks on this.
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-09 12:06 David Geier <geidav.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 2 replies; 51+ messages in thread
From: David Geier @ 2026-01-09 12:06 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
Hi Heikki!
Thanks for looking at the patch set.
On 06.01.2026 18:00, Heikki Linnakangas wrote:
> On 05/01/2026 17:01, David Geier wrote:
>> - v1-0002-Optimized-comparison-functions.patch: Use FunctionCallInvoke()
>> instead of FunctionCall2Coll(). This saves a bunch of per-comparison
>> setup code, such as calling InitFunctionCallInfoData().
>
> You lose the check for NULL result with this. That's probably still
> worth checking.
It seems like existing code where all args are not null, has that safety
check. Added it for consistency.
>> - v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
>> ginExtractEntries() deduplicates and sorts the entries returned from the
>> extract value function. In case of pg_trgm, that is completely redundant
>> because the trigrams are already deduplicated and sorted. The current
>> version of this patch is just to demonstrate the potential. We need to
>> think about what we want here. Ideally, we would require the extraction
>> function to provide the entries deduplicated and sorted. Alternatively,
>> we could indicate to ginExtractEntries() if the entries are already
>> deduplicated and sorted. If we don't want to alter the signature of the
>> extract value function, we could e.g. use the MSB of the nentries
>> argument.
>
> Yeah, this seems wrong as it is. You're assuming that if the extract
> function returns nullFlags == NULL, the array is already sorted and
> deduped.
As said, that was just for demonstration purposes of the possible gains.
I've changed the code now such that the extractValue function of the GIN
index can indicate via the third argument uniqueAndSorted, if the
returned keys are already unique and sorted.
Unfortunately, it seems like this patch regresses performance. See
measurements below. I haven't had the time to investigate why that is.
It's pretty counter intuitive, given that this patch effectively only
removes code. Maybe you could re-test patch 0004 and share your runtimes?
>> v1-0007-Faster-qunique-comparator.patch: qunique() doesn't require a
>> full sort comparator (-1 = less, 0 = equal, 1 = greater) but only a
>> equal/unequal comparator (e.g. 0 = equal and 1 = unequal). The same
>> optimization can be done in plenty of other places in our code base.
>> Likely, in most of them the gains are insignificant.
>
> Makes sense. I'm a little disappointed the compiler won't do that
> optimization for us..
I thought the same.
>
> Perhaps we should introduce a new qunique_eq() function with a different
> callback signature:
>
> /* like qunique(), but the callback function returns true/false rather
> than int */
> static inline size_t
> qunique_eq(void *array, size_t elements, size_t width,
> bool (*equal) (const void *, const void *))
>
I would prefer to change qunique() instead. That would enforce using an
adequate comparison function from the get go. There are only ~15 calls
to qunique(). So refactoring this should also be a fairly small patch. I
can do that if there's agreement for that approach.
>> v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch: Typically lots
>> of text is actually ASCII. Hence, we provide a fast path for this case
>> which is exercised if the MSB of the current character is unset.
>
> This uses pg_ascii_tolower() when for ASCII characters when built with
> the IGNORECASE. I don't think that's correct, if the proper collation
> would do something more complicated for than what pg_ascii_tolower() does.
Oh, that's evil. I had tested that specifically. But it only worked
because the code in master uses str_tolower() with
DEFAULT_COLLATION_OID. So using a different locale like in the following
example does something different than when creating a database with the
same locale.
postgres=# select lower('III' COLLATE "tr_TR");
lower
-------
ııı
postgres=# select show_trgm('III' COLLATE "tr_TR");
show_trgm
-------------------------
{" i"," ii","ii ",iii}
(1 row)
But when using tr_TR as default locale of the database the following
happens:
postgres=# select lower('III' COLLATE "tr_TR");
lower
-------
ııı
postgres=# select show_trgm('III');sü
show_trgm
---------------------------------------
{0xbbd8dd,0xf26fab,0xf31e1a,0x2af4f1}
I'm wondering if that's intentional to begin with. Shouldn't the code
instead pass PG_GET_COLLATION() to str_tolower()? Might require some
research to see how other index types handle locales.
Coming back to the original problem: the lengthy comment at the top of
pg_locale_libc.c, suggests that in some cases ASCII characters are
handled the pg_ascii_tolower() way for the default locale. See for
example tolower_libc_mb(). So a character by character conversion using
that function will yield a different result than strlower_libc_mb(). I'm
wondering why that is.
Anyways, we could limit the optimization to only kick in when the used
locale follows the same rules as pg_ascii_tolower(). We could test that
when creating the locale and store that info in pg_locale_struct.
Thoughts?
> Did you measure how big is the impact from each individual patch?
> Patches 1 and 2 seem pretty much ready to be committed, but I wonder if
> they make any difference on their own.
Here is the impact of each patch. I ran again CREATE INDEX three times
and took the fastest run. The run of each patch includes all previous
patches as well. For example, the timings for patch 0003 were measured
with a binary that also had patch 0002 and 0001 applied. To get the
impact of each patch in isolation, the delta to the previous run was taken.
Code | movies |delta | lineitem | delta
------------------------------------|--------|-------|------------------
master | 10,311 | 0 | 256,986 | 0
v1-0001-Inline-ginCompareAttEntries | 9,694 | 617 | 239,778 | 17,208
v1-0002-Optimized-comparison-func | 9,510 | 184 | 238,094 | 1,684
v1-0003-Use-sort_template.h | 8,661 | 849 | 231,190 | 6,904
v1-0004-Avoid-dedup-and-sort-in | 9,305 | -644 | 232,472 | -1,282
v1-0005-Make-btint4cmp-branchless | 8,240 | 1,065 | 228,387 | 4,085
v1-0006-Use-radix-sort | 6,976 | 1,264 | 207,687 | 20,700
v1-0007-Faster-qunique-comparator | 5,911 | 1,065 | 203,744 | 3,943
v1-0008-Add-ASCII-fastpath | 3,409 | 2,502 | 161,469 | 42,275
Attached is v2 of the patch set with the aforementioned changes. I've
also fixed the white space errors in 0003, 0004 and 0008, as reported by
Kirill.
--
David Geier
Attachments:
[text/x-patch] v2-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch (4.0K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/2-v2-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch)
download | inline diff:
From 00c75bd58b7982c8bfe62d1260e9366766bc7f34 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Fri, 14 Nov 2025 11:37:40 +0100
Subject: [PATCH v2 8/8] Add ASCII fastpath to generate_trgm_only()
---
contrib/pg_trgm/trgm_op.c | 124 ++++++++++++++++++++------------------
1 file changed, 65 insertions(+), 59 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 39b586f5b9a..d2087b3a45e 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,32 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-/*
- * Finds first word in string, returns pointer to the word,
- * endword points to the character after word
- */
-static char *
-find_word(char *str, int lenstr, char **endword, int *charlen)
-{
- char *beginword = str;
-
- while (beginword - str < lenstr && !ISWORDCHR(beginword))
- beginword += pg_mblen(beginword);
-
- if (beginword - str >= lenstr)
- return NULL;
-
- *endword = beginword;
- *charlen = 0;
- while (*endword - str < lenstr && ISWORDCHR(*endword))
- {
- *endword += pg_mblen(*endword);
- (*charlen)++;
- }
-
- return beginword;
-}
-
/*
* Reduce a trigram (three possibly multi-byte characters) to a trgm,
* which is always exactly three bytes. If we have three single-byte
@@ -337,58 +311,90 @@ make_trigrams(trgm *tptr, char *str, int bytelen, int charlen)
static int
generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
{
- trgm *tptr;
- char *buf;
- int charlen,
- bytelen;
- char *bword,
- *eword;
+ trgm *tptr = trg;
+ char *buf;
if (slen + LPADDING + RPADDING < 3 || slen == 0)
return 0;
- tptr = trg;
-
- /* Allocate a buffer for case-folded, blank-padded words */
- buf = (char *) palloc(slen * pg_database_encoding_max_length() + 4);
+ buf = palloc_array(char, slen * pg_database_encoding_max_length() + 4 + 1);
+ memset(buf, ' ', LPADDING);
- if (LPADDING > 0)
+ for (int i = 0; i < slen; )
{
- *buf = ' ';
- if (LPADDING > 1)
- *(buf + 1) = ' ';
- }
+ int num_bytes = LPADDING;
+ int num_chars = LPADDING;
+ char *word;
- eword = str;
- while ((bword = find_word(eword, slen - (eword - str), &eword, &charlen)) != NULL)
- {
+ /* Extract next word */
+ while (i < slen)
+ {
+ if ((str[i] & 0x80) == 0) /* Fast path for ASCII-only */
+ {
+ if (isalnum(str[i]))
+ {
#ifdef IGNORECASE
- bword = str_tolower(bword, eword - bword, DEFAULT_COLLATION_OID);
- bytelen = strlen(bword);
+ buf[num_bytes++] = pg_ascii_tolower(str[i++]);
#else
- bytelen = eword - bword;
+ buf[num_bytes++] = str[i++];
#endif
+ }
+ else
+ {
+ i++;
+ break;
+ }
+ }
+ else
+ {
+ const int mblen = pg_mblen(str + i);
+ Assert(mblen >= 2); /* Otherwise, it would be ASCII */
+
+ if (ISWORDCHR(str + i))
+ {
+ memcpy(buf + num_bytes, str + i, mblen);
+ num_bytes += mblen;
+ i += mblen;
+ }
+ else
+ {
+ i += mblen;
+ break;
+ }
+ }
+
+ num_chars++;
+ }
- memcpy(buf + LPADDING, bword, bytelen);
+ if (num_chars > LPADDING)
+ {
+ memset(buf + num_bytes, ' ', RPADDING);
+ num_bytes += RPADDING;
+ num_chars += RPADDING;
+ word = buf;
#ifdef IGNORECASE
- pfree(bword);
+ if (num_chars != num_bytes)
+ {
+ word = str_tolower(buf, num_bytes, DEFAULT_COLLATION_OID);
+ num_bytes = strlen(word); /* String can get shorter from lower-casing */
+ }
#endif
- buf[LPADDING + bytelen] = ' ';
- buf[LPADDING + bytelen + 1] = ' ';
+ if (bounds)
+ bounds[tptr - trg] |= TRGM_BOUND_LEFT;
+
+ tptr = make_trigrams(tptr, word, num_bytes, num_chars);
+
+ if (bounds)
+ bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
- /* Calculate trigrams marking their bounds if needed */
- if (bounds)
- bounds[tptr - trg] |= TRGM_BOUND_LEFT;
- tptr = make_trigrams(tptr, buf, bytelen + LPADDING + RPADDING,
- charlen + LPADDING + RPADDING);
- if (bounds)
- bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
+ if (word != buf)
+ pfree(word);
+ }
}
pfree(buf);
-
return tptr - trg;
}
--
2.51.0
[text/x-patch] v2-0007-Faster-qunique-comparator.patch (1.9K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/3-v2-0007-Faster-qunique-comparator.patch)
download | inline diff:
From 457a3d0d57a834a80237d628d353408f3e2f4378 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 12 Nov 2025 14:27:13 +0100
Subject: [PATCH v2 7/8] Faster qunique() comparator
---
contrib/pg_trgm/trgm_op.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 039c273f6a1..39b586f5b9a 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -139,6 +139,14 @@ CMPTRGM_UNSIGNED(const void *a, const void *b)
: CMPPCHAR_UNS(a, b, 2));
}
+static inline int
+CMPTRGM_EQ(const void *a, const void *b)
+{
+ char *aa = (char *)a;
+ char *bb = (char *)b;
+ return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
+}
+
/*
* This gets called on the first call. It replaces the function pointer so
* that subsequent calls are routed directly to the chosen implementation.
@@ -482,15 +490,11 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1038,15 +1042,11 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v2-0006-Use-radix-sort.patch (2.9K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/4-v2-0006-Use-radix-sort.patch)
download | inline diff:
From 447ce313602e768f29c43d4a5359f4c909b8db46 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v2 6/8] Use radix sort
---
contrib/pg_trgm/trgm_op.c | 64 +++++++++++++++++++++++++++++++++------
1 file changed, 54 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 6af120fa1ad..039c273f6a1 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -155,14 +155,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
}
/* Define our specialized sort function name */
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
@@ -392,6 +384,58 @@ generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
return tptr - trg;
}
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ Assert(to == buffer);
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
+
/*
* Guard against possible overflow in the palloc requests below. (We
* don't worry about the additive constants, since palloc can detect
@@ -439,7 +483,7 @@ generate_trgm(char *str, int slen)
{
if (GetDefaultCharSignedness())
{
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
}
else
@@ -995,7 +1039,7 @@ generate_wildcard_trgm(const char *str, int slen)
{
if (GetDefaultCharSignedness())
{
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
}
else
--
2.51.0
[text/x-patch] v2-0005-Make-btint4cmp-branchless.patch (1.0K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/5-v2-0005-Make-btint4cmp-branchless.patch)
download | inline diff:
From c68aa700a245f1eb05f701a227776e24d0c0afe7 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v2 5/8] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index bffc4b7709c..5ae27c22621 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -60,6 +60,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -202,12 +203,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
[text/x-patch] v2-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch (4.9K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/6-v2-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch)
download | inline diff:
From 0cdc87640cf2a2d8c946aff1669706f99280c555 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 14:40:37 +0100
Subject: [PATCH v2 4/8] Avoid dedup and sort in ginExtractEntries
---
contrib/pg_trgm/trgm_gin.c | 2 ++
doc/src/sgml/gin.sgml | 7 ++++++-
src/backend/access/gin/ginutil.c | 32 +++++++++++++++++++-------------
3 files changed, 27 insertions(+), 14 deletions(-)
diff --git a/contrib/pg_trgm/trgm_gin.c b/contrib/pg_trgm/trgm_gin.c
index 66ff6adde99..862e650efec 100644
--- a/contrib/pg_trgm/trgm_gin.c
+++ b/contrib/pg_trgm/trgm_gin.c
@@ -36,10 +36,12 @@ gin_extract_value_trgm(PG_FUNCTION_ARGS)
{
text *val = (text *) PG_GETARG_TEXT_PP(0);
int32 *nentries = (int32 *) PG_GETARG_POINTER(1);
+ bool *uniqueAndSorted = (bool *) PG_GETARG_POINTER(3);
Datum *entries = NULL;
TRGM *trg;
int32 trglen;
+ *uniqueAndSorted = true;
*nentries = 0;
trg = generate_trgm(VARDATA_ANY(val), VARSIZE_ANY_EXHDR(val));
diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml
index 82410b1fbdf..b96478731f8 100644
--- a/doc/src/sgml/gin.sgml
+++ b/doc/src/sgml/gin.sgml
@@ -167,7 +167,7 @@
<variablelist>
<varlistentry>
<term><function>Datum *extractValue(Datum itemValue, int32 *nkeys,
- bool **nullFlags)</function></term>
+ bool **nullFlags, bool *uniqueAndSorted)</function></term>
<listitem>
<para>
Returns a palloc'd array of keys given an item to be indexed. The
@@ -177,6 +177,11 @@
<literal>*nullFlags</literal>, and set these null flags as needed.
<literal>*nullFlags</literal> can be left <symbol>NULL</symbol> (its initial value)
if all keys are non-null.
+ If the returned keys do not contain duplicates and are sorted w.r.t. the comparison
+ function of the GIN type's operator class, store <symbol>true</symbol> in
+ <literal>uniqueAndSorted</literal>. <literal>uniqueAndSorted</literal> can be left
+ <symbol>false</symbol> (its initial value) if the keys are either unsorted or contain
+ duplicates. In that case, duplicate removal and sorting is performed by the GIN index.
The return value can be <symbol>NULL</symbol> if the item contains no keys.
</para>
</listitem>
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 75a18f457bc..22b588483d0 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -464,6 +464,7 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
{
Datum *entries;
bool *nullFlags;
+ bool uniqueAndSorted = false;
int32 i;
/*
@@ -483,11 +484,12 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
/* OK, call the opclass's extractValueFn */
nullFlags = NULL; /* in case extractValue doesn't set it */
entries = (Datum *)
- DatumGetPointer(FunctionCall3Coll(&ginstate->extractValueFn[attnum - 1],
+ DatumGetPointer(FunctionCall4Coll(&ginstate->extractValueFn[attnum - 1],
ginstate->supportCollation[attnum - 1],
value,
PointerGetDatum(nentries),
- PointerGetDatum(&nullFlags)));
+ PointerGetDatum(&nullFlags),
+ PointerGetDatum(&uniqueAndSorted)));
/*
* Generate a placeholder if the item contained no keys.
@@ -502,13 +504,6 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
return entries;
}
- /*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
- */
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
-
/*
* If there's more than one key, sort and unique-ify.
*
@@ -516,11 +511,18 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
+ if (*nentries > 1 && !uniqueAndSorted)
{
keyEntryData *keydata;
cmpEntriesArg arg;
+ /*
+ * If the extractValueFn didn't create a nullFlags array, create one,
+ * assuming that everything's non-null.
+ */
+ if (nullFlags == NULL)
+ nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
+
keydata = palloc_array(keyEntryData, *nentries);
for (i = 0; i < *nentries; i++)
{
@@ -568,9 +570,13 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
/*
* Create GinNullCategory representation from nullFlags.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY is 0");
+ *categories = palloc0_array(GinNullCategory, *nentries);
+
+ if (nullFlags != NULL)
+ for (i = 0; i < *nentries; i++)
+ if (nullFlags[i])
+ (*categories)[i] = GIN_CAT_NULL_KEY;
return entries;
}
--
2.51.0
[text/x-patch] v2-0003-Use-sort_template.h.patch (2.6K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/7-v2-0003-Use-sort_template.h.patch)
download | inline diff:
From c38f3517d05d3cefa0fc6d994f4e8f6ac14273d9 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:11 +0100
Subject: [PATCH v2 3/8] Use sort_template.h
---
contrib/pg_trgm/trgm_op.c | 47 ++++++++++++++++++++++++++++++---------
1 file changed, 37 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 81182a15e07..6af120fa1ad 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -154,6 +154,23 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
+/* Define our specialized sort function name */
+#define ST_SORT trigram_qsort_signed
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
+#define ST_SORT trigram_qsort_unsigned
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
/*
* Deprecated function.
* Use "pg_trgm.similarity_threshold" GUC variable instead of this function.
@@ -209,12 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-static int
-comp_trgm(const void *a, const void *b)
-{
- return CMPTRGM(a, b);
-}
-
/*
* Finds first word in string, returns pointer to the word,
* endword points to the character after word
@@ -426,8 +437,16 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -974,8 +993,16 @@ generate_wildcard_trgm(const char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v2-0002-Optimized-comparison-functions.patch (4.1K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/8-v2-0002-Optimized-comparison-functions.patch)
download | inline diff:
From ae80c2dc190493f8c9811c1949c7ebced265abd5 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:01 +0100
Subject: [PATCH v2 2/8] Optimized comparison functions
---
src/backend/access/gin/ginutil.c | 23 ++++++++++++++++-------
src/include/access/gin_private.h | 19 +++++++++++++++----
2 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d205093e21d..75a18f457bc 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -114,6 +114,7 @@ initGinState(GinState *state, Relation index)
for (i = 0; i < origTupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(origTupdesc, i);
+ FunctionCallInfoBaseData *fci = &state->compareFnCallInfo[i].fcinfo;
if (state->oneCol)
state->tupdesc[i] = state->origTupdesc;
@@ -222,6 +223,10 @@ initGinState(GinState *state, Relation index)
state->supportCollation[i] = index->rd_indcollation[i];
else
state->supportCollation[i] = DEFAULT_COLLATION_OID;
+
+ InitFunctionCallInfoData(*fci, &state->compareFn[i], 2, state->supportCollation[i], NULL, NULL);
+ fci->args[0].isnull = false;
+ fci->args[1].isnull = false;
}
}
@@ -402,8 +407,7 @@ typedef struct
typedef struct
{
- FmgrInfo *cmpDatumFunc;
- Oid collation;
+ FunctionCallInfoBaseData *cmpFuncInfo;
bool haveDups;
} cmpEntriesArg;
@@ -425,9 +429,15 @@ cmpEntries(const void *a, const void *b, void *arg)
else if (bb->isnull)
res = -1; /* not-NULL "<" NULL */
else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ {
+ FunctionCallInfo fci = data->cmpFuncInfo;
+ fci->args[0].value = aa->datum;
+ fci->args[1].value = bb->datum;
+ res = DatumGetInt32(FunctionCallInvoke(fci));
+
+ if (fci->isnull)
+ elog(ERROR, "function %u returned NULL", fci->flinfo->fn_oid);
+ }
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -518,8 +528,7 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
keydata[i].isnull = nullFlags[i];
}
- arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
- arg.collation = ginstate->supportCollation[attnum - 1];
+ arg.cmpFuncInfo = &ginstate->compareFnCallInfo[attnum - 1].fcinfo;
arg.haveDups = false;
qsort_arg(keydata, *nentries, sizeof(keyEntryData),
cmpEntries, &arg);
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index e155045ce8a..7cf19c8a5dc 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -51,6 +51,11 @@ typedef struct GinOptions
#define GIN_SHARE BUFFER_LOCK_SHARE
#define GIN_EXCLUSIVE BUFFER_LOCK_EXCLUSIVE
+typedef union CompareFuncCallInfoData
+{
+ FunctionCallInfoBaseData fcinfo;
+ char fcinfo_data[SizeForFunctionCallInfo(2)];
+} CompareFuncCallInfoData;
/*
* GinState: working data structure describing the index being worked on
@@ -77,6 +82,10 @@ typedef struct GinState
/*
* Per-index-column opclass support functions
*/
+
+
+ CompareFuncCallInfoData compareFnCallInfo[INDEX_MAX_KEYS];
+
FmgrInfo compareFn[INDEX_MAX_KEYS];
FmgrInfo extractValueFn[INDEX_MAX_KEYS];
FmgrInfo extractQueryFn[INDEX_MAX_KEYS];
@@ -504,6 +513,8 @@ ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
Datum a, GinNullCategory categorya,
Datum b, GinNullCategory categoryb)
{
+ FunctionCallInfoBaseData *fci;
+
/* if not of same null category, sort by that first */
if (categorya != categoryb)
return (categorya < categoryb) ? -1 : 1;
@@ -512,10 +523,10 @@ ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
if (categorya != GIN_CAT_NORM_KEY)
return 0;
- /* both not null, so safe to call the compareFn */
- return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
- ginstate->supportCollation[attnum - 1],
- a, b));
+ fci = &ginstate->compareFnCallInfo[attnum - 1].fcinfo;
+ fci->args[0].value = a;
+ fci->args[1].value = b;
+ return DatumGetInt32(FunctionCallInvoke(fci));
}
/*
--
2.51.0
[text/x-patch] v2-0001-Inline-ginCompareAttEntries.patch (4.0K, ../../e5dd01c6-c469-405d-aea2-feca0b2dc34d@gmail.com/9-v2-0001-Inline-ginCompareAttEntries.patch)
download | inline diff:
From a484e7c69bec62474b041eb4e533601e4883dab3 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Thu, 6 Nov 2025 09:42:27 +0100
Subject: [PATCH v2 1/8] Inline ginCompareAttEntries
---
src/backend/access/gin/ginutil.c | 38 ---------------------------
src/include/access/gin_private.h | 44 +++++++++++++++++++++++++++-----
2 files changed, 38 insertions(+), 44 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index a546cac18d3..d205093e21d 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -387,44 +387,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Compare two keys of the same index column
- */
-int
-ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
- Datum a, GinNullCategory categorya,
- Datum b, GinNullCategory categoryb)
-{
- /* if not of same null category, sort by that first */
- if (categorya != categoryb)
- return (categorya < categoryb) ? -1 : 1;
-
- /* all null items in same category are equal */
- if (categorya != GIN_CAT_NORM_KEY)
- return 0;
-
- /* both not null, so safe to call the compareFn */
- return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
- ginstate->supportCollation[attnum - 1],
- a, b));
-}
-
-/*
- * Compare two keys of possibly different index columns
- */
-int
-ginCompareAttEntries(GinState *ginstate,
- OffsetNumber attnuma, Datum a, GinNullCategory categorya,
- OffsetNumber attnumb, Datum b, GinNullCategory categoryb)
-{
- /* attribute number is the first sort key */
- if (attnuma != attnumb)
- return (attnuma < attnumb) ? -1 : 1;
-
- return ginCompareEntries(ginstate, attnuma, a, categorya, b, categoryb);
-}
-
-
/*
* Support for sorting key datums in ginExtractEntries
*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index b33f7cec5b4..e155045ce8a 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -97,12 +97,6 @@ extern Buffer GinNewBuffer(Relation index);
extern void GinInitBuffer(Buffer b, uint32 f);
extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
-extern int ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
- Datum a, GinNullCategory categorya,
- Datum b, GinNullCategory categoryb);
-extern int ginCompareAttEntries(GinState *ginstate,
- OffsetNumber attnuma, Datum a, GinNullCategory categorya,
- OffsetNumber attnumb, Datum b, GinNullCategory categoryb);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
int32 *nentries, GinNullCategory **categories);
@@ -502,6 +496,44 @@ ginCompareItemPointers(ItemPointer a, ItemPointer b)
return pg_cmp_u64(ia, ib);
}
+/*
+ * Compare two keys of the same index column
+ */
+static inline int
+ginCompareEntries(GinState *ginstate, OffsetNumber attnum,
+ Datum a, GinNullCategory categorya,
+ Datum b, GinNullCategory categoryb)
+{
+ /* if not of same null category, sort by that first */
+ if (categorya != categoryb)
+ return (categorya < categoryb) ? -1 : 1;
+
+ /* all null items in same category are equal */
+ if (categorya != GIN_CAT_NORM_KEY)
+ return 0;
+
+ /* both not null, so safe to call the compareFn */
+ return DatumGetInt32(FunctionCall2Coll(&ginstate->compareFn[attnum - 1],
+ ginstate->supportCollation[attnum - 1],
+ a, b));
+}
+
+/*
+ * Compare two keys of possibly different index columns
+ */
+static inline int
+ginCompareAttEntries(GinState *ginstate,
+ OffsetNumber attnuma, Datum a, GinNullCategory categorya,
+ OffsetNumber attnumb, Datum b, GinNullCategory categoryb)
+{
+ /* attribute number is the first sort key */
+ if (attnuma != attnumb)
+ return (attnuma < attnumb) ? -1 : 1;
+
+ return ginCompareEntries(ginstate, attnuma, a, categorya, b, categoryb);
+
+}
+
extern int ginTraverseLock(Buffer buffer, bool searchMode);
#endif /* GIN_PRIVATE_H */
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-09 12:10 David Geier <geidav.pg@gmail.com>
parent: Kirill Reshke <reshkekirill@gmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: David Geier @ 2026-01-09 12:10 UTC (permalink / raw)
To: Kirill Reshke <reshkekirill@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: pgsql-hackers
Hi!
> Hi!
> patches 0003, 0004 & 0008 applies with whitespace errors.
>
I've fixed the white space errors. See v2 of the patch set in [1].
> I did run benchmarks of my VM using your data. v1-0001 solely improves
> runtime by 4-5%. v2-0002 does not show runtime improvement for me.
> With parallel GIN build, performance gains are 2-3 % for 2 workers and
> not noticeable after that.
>
> I will try to run some more benchmarks on this.
Thanks. I've also included the delta for each patch in [1]. I would be
curious what you measure, especially for patch 0004, where I curiously
measured a regression.
[1]
https://www.postgresql.org/message-id/e5dd01c6-c469-405d-aea2-feca0b2dc34d%40gmail.com
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-09 18:36 Heikki Linnakangas <hlinnaka@iki.fi>
parent: David Geier <geidav.pg@gmail.com>
1 sibling, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-01-09 18:36 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; pgsql-hackers
On 09/01/2026 14:06, David Geier wrote:
> On 06.01.2026 18:00, Heikki Linnakangas wrote:
>> On 05/01/2026 17:01, David Geier wrote:
>>> - v1-0002-Optimized-comparison-functions.patch: Use FunctionCallInvoke()
>>> instead of FunctionCall2Coll(). This saves a bunch of per-comparison
>>> setup code, such as calling InitFunctionCallInfoData().
>>
>> You lose the check for NULL result with this. That's probably still
>> worth checking.
>
> It seems like existing code where all args are not null, has that safety
> check. Added it for consistency.
>
>>> - v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
>>> ginExtractEntries() deduplicates and sorts the entries returned from the
>>> extract value function. In case of pg_trgm, that is completely redundant
>>> because the trigrams are already deduplicated and sorted. The current
>>> version of this patch is just to demonstrate the potential. We need to
>>> think about what we want here. Ideally, we would require the extraction
>>> function to provide the entries deduplicated and sorted. Alternatively,
>>> we could indicate to ginExtractEntries() if the entries are already
>>> deduplicated and sorted. If we don't want to alter the signature of the
>>> extract value function, we could e.g. use the MSB of the nentries
>>> argument.
>>
>> Yeah, this seems wrong as it is. You're assuming that if the extract
>> function returns nullFlags == NULL, the array is already sorted and
>> deduped.
>
> As said, that was just for demonstration purposes of the possible gains.
> I've changed the code now such that the extractValue function of the GIN
> index can indicate via the third argument uniqueAndSorted, if the
> returned keys are already unique and sorted.
>
> Unfortunately, it seems like this patch regresses performance. See
> measurements below. I haven't had the time to investigate why that is.
> It's pretty counter intuitive, given that this patch effectively only
> removes code. Maybe you could re-test patch 0004 and share your runtimes?
Looking at 0001 and 0004 patches and ginExtractEntries(), the way
ginExtractEntries() handles NULLs looks a little inefficient. It treats
NULLs as proper entries, passing them through qsort() for deduplication
and comparing them with cmpEntries(). But the end result is always that
if the opclass's extractValueFn() function returned any NULLs, then
there's exctly one GIN_CAT_NULL_KEY as the last entry of the result
array. Surely we could be smarter about how we accomplish that. Your
0004 patch eliminates the deduplication overhead altogether, which is
great of course, but the point remains for when we still need the
deduplication.
Attached is an attempt at that. It avoids the null-checks in
cmpEntries(), saving a few cycles. That's drowned in noise with your
test cases, but with the attached test case with int arrays, I'm seeing
a 1-2 % gain. That's not much, but I think it's still worth doing
because it makes the code a little simpler too, IMHO. (I didn't test it
together with the rest of your patches.)
>> Perhaps we should introduce a new qunique_eq() function with a different
>> callback signature:
>>
>> /* like qunique(), but the callback function returns true/false rather
>> than int */
>> static inline size_t
>> qunique_eq(void *array, size_t elements, size_t width,
>> bool (*equal) (const void *, const void *))
>>
>
> I would prefer to change qunique() instead. That would enforce using an
> adequate comparison function from the get go. There are only ~15 calls
> to qunique(). So refactoring this should also be a fairly small patch. I
> can do that if there's agreement for that approach.
Works for me.
At quick glance, most if not all of the qunique() calls call qsort()
just before qunique(). I wonder if we should have a single "sort and
deduplicate" function, instead. It could perhaps do some deduplication
"on the go", or other optimizations.
>> Did you measure how big is the impact from each individual patch?
>> Patches 1 and 2 seem pretty much ready to be committed, but I wonder if
>> they make any difference on their own.
>
> Here is the impact of each patch. I ran again CREATE INDEX three times
> and took the fastest run. The run of each patch includes all previous
> patches as well. For example, the timings for patch 0003 were measured
> with a binary that also had patch 0002 and 0001 applied. To get the
> impact of each patch in isolation, the delta to the previous run was taken.
>
> Code | movies |delta | lineitem | delta
> ------------------------------------|--------|-------|------------------
> master | 10,311 | 0 | 256,986 | 0
> v1-0001-Inline-ginCompareAttEntries | 9,694 | 617 | 239,778 | 17,208
> v1-0002-Optimized-comparison-func | 9,510 | 184 | 238,094 | 1,684
> v1-0003-Use-sort_template.h | 8,661 | 849 | 231,190 | 6,904
> v1-0004-Avoid-dedup-and-sort-in | 9,305 | -644 | 232,472 | -1,282
> v1-0005-Make-btint4cmp-branchless | 8,240 | 1,065 | 228,387 | 4,085
> v1-0006-Use-radix-sort | 6,976 | 1,264 | 207,687 | 20,700
> v1-0007-Faster-qunique-comparator | 5,911 | 1,065 | 203,744 | 3,943
> v1-0008-Add-ASCII-fastpath | 3,409 | 2,502 | 161,469 | 42,275
>
> Attached is v2 of the patch set with the aforementioned changes. I've
> also fixed the white space errors in 0003, 0004 and 0008, as reported by
> Kirill.
Thanks, I pushed patch 0001 now, that's a simple and clear win.
- Heikki
Attachments:
[text/x-patch] 0001-Make-the-deduplication-in-ginExtractEntries-a-little.patch (7.5K, ../../6d16b6bd-a1ff-4469-aefb-a1c8274e561a@iki.fi/2-0001-Make-the-deduplication-in-ginExtractEntries-a-little.patch)
download | inline diff:
From a0348e8dedb9a7ee53af8f88adbba0e5aeff577d Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 9 Jan 2026 19:31:19 +0200
Subject: [PATCH 1/1] Make the deduplication in ginExtractEntries() a little
faster
The idea is to remove NULLs from the array first, and use qsort to
deduplicate only the non-NULL items. That simplifies the comparison
function a little.
---
src/backend/access/gin/ginutil.c | 139 +++++++++++++------------------
src/include/access/gin_private.h | 2 +-
2 files changed, 61 insertions(+), 80 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d205093e21d..29966b9d05b 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -387,19 +387,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Support for sorting key datums in ginExtractEntries
- *
- * Note: we only have to worry about null and not-null keys here;
- * ginExtractEntries never generates more than one placeholder null,
- * so it doesn't have to sort those.
- */
-typedef struct
-{
- Datum datum;
- bool isnull;
-} keyEntryData;
-
typedef struct
{
FmgrInfo *cmpDatumFunc;
@@ -410,24 +397,14 @@ typedef struct
static int
cmpEntries(const void *a, const void *b, void *arg)
{
- const keyEntryData *aa = (const keyEntryData *) a;
- const keyEntryData *bb = (const keyEntryData *) b;
+ const Datum *aa = (const Datum *) a;
+ const Datum *bb = (const Datum *) b;
cmpEntriesArg *data = (cmpEntriesArg *) arg;
int res;
- if (aa->isnull)
- {
- if (bb->isnull)
- res = 0; /* NULL "=" NULL */
- else
- res = 1; /* NULL ">" not-NULL */
- }
- else if (bb->isnull)
- res = -1; /* not-NULL "<" NULL */
- else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
+ data->collation,
+ *aa, *bb));
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -450,11 +427,13 @@ cmpEntries(const void *a, const void *b, void *arg)
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories)
+ int32 *nentries_p, GinNullCategory **categories_p)
{
Datum *entries;
bool *nullFlags;
- int32 i;
+ GinNullCategory *categories;
+ bool hasNull;
+ int32 nentries;
/*
* We don't call the extractValueFn on a null item. Instead generate a
@@ -462,42 +441,60 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
*/
if (isNull)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_NULL_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_NULL_ITEM;
return entries;
}
/* OK, call the opclass's extractValueFn */
nullFlags = NULL; /* in case extractValue doesn't set it */
+ nentries = 0;
entries = (Datum *)
DatumGetPointer(FunctionCall3Coll(&ginstate->extractValueFn[attnum - 1],
ginstate->supportCollation[attnum - 1],
value,
- PointerGetDatum(nentries),
+ PointerGetDatum(&nentries),
PointerGetDatum(&nullFlags)));
/*
* Generate a placeholder if the item contained no keys.
*/
- if (entries == NULL || *nentries <= 0)
+ if (entries == NULL || nentries <= 0)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_EMPTY_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_EMPTY_ITEM;
return entries;
}
/*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
+ * Scan the items for any NULLs. All NULLs are considered equal, so we
+ * just need to check and remember if there are any. We remove them from
+ * the array here, and if necessary, put back one NULL entry to represent
+ * them all after deduplication.
*/
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
+ hasNull = false;
+ if (nullFlags)
+ {
+ int32 numNonNulls = 0;
+
+ for (int32 i = 0; i < nentries; i++)
+ {
+ if (nullFlags[i])
+ hasNull = true;
+ else
+ {
+ entries[numNonNulls] = entries[i];
+ numNonNulls++;
+ }
+ }
+ nentries = numNonNulls;
+ }
/*
* If there's more than one key, sort and unique-ify.
@@ -506,63 +503,47 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
+ if (nentries > 1)
{
- keyEntryData *keydata;
cmpEntriesArg arg;
- keydata = palloc_array(keyEntryData, *nentries);
- for (i = 0; i < *nentries; i++)
- {
- keydata[i].datum = entries[i];
- keydata[i].isnull = nullFlags[i];
- }
-
arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
arg.collation = ginstate->supportCollation[attnum - 1];
arg.haveDups = false;
- qsort_arg(keydata, *nentries, sizeof(keyEntryData),
+ qsort_arg(entries, nentries, sizeof(Datum),
cmpEntries, &arg);
if (arg.haveDups)
{
/* there are duplicates, must get rid of 'em */
- int32 j;
+ int32 j = 1;
- entries[0] = keydata[0].datum;
- nullFlags[0] = keydata[0].isnull;
- j = 1;
- for (i = 1; i < *nentries; i++)
+ for (int32 i = 1; i < nentries; i++)
{
- if (cmpEntries(&keydata[i - 1], &keydata[i], &arg) != 0)
- {
- entries[j] = keydata[i].datum;
- nullFlags[j] = keydata[i].isnull;
- j++;
- }
+ if (cmpEntries(&entries[i - 1], &entries[i], &arg) != 0)
+ entries[j++] = entries[i];
}
- *nentries = j;
+ nentries = j;
}
- else
- {
- /* easy, no duplicates */
- for (i = 0; i < *nentries; i++)
- {
- entries[i] = keydata[i].datum;
- nullFlags[i] = keydata[i].isnull;
- }
- }
-
- pfree(keydata);
}
/*
- * Create GinNullCategory representation from nullFlags.
+ * Create GinNullCategory representation.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ categories = (GinNullCategory *) palloc((nentries + (hasNull ? 1 : 0)) * sizeof(GinNullCategory));
+ for (int32 i = 0; i < nentries; i++)
+ categories[i] = GIN_CAT_NORM_KEY;
+
+ /* Put back a NULL entry, if there were any */
+ if (hasNull)
+ {
+ entries[nentries] = (Datum) 0;
+ categories[nentries] = GIN_CAT_NULL_KEY;
+ nentries++;
+ }
+ *nentries_p = nentries;
+ *categories_p = categories;
return entries;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index e155045ce8a..1e97e6cb3c9 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -99,7 +99,7 @@ extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories);
+ int32 *nentries_p, GinNullCategory **categories_p);
extern OffsetNumber gintuple_get_attrnum(GinState *ginstate, IndexTuple tuple);
extern Datum gintuple_get_key(GinState *ginstate, IndexTuple tuple,
--
2.47.3
[application/sql] test-gin-array.sql (341B, ../../6d16b6bd-a1ff-4469-aefb-a1c8274e561a@iki.fi/3-test-gin-array.sql)
download
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-09 21:02 David Geier <geidav.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-01-09 21:02 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
On 09.01.2026 19:36, Heikki Linnakangas wrote:
>>>> - v1-0004-Avoid-dedup-and-sort-in-ginExtractEntries.patch
>>>> ginExtractEntries() deduplicates and sorts the entries returned from
>>>> the
>>>> extract value function. In case of pg_trgm, that is completely
>>>> redundant
>>>> because the trigrams are already deduplicated and sorted. The current
>>>> version of this patch is just to demonstrate the potential. We need to
>>>> think about what we want here. Ideally, we would require the extraction
>>>> function to provide the entries deduplicated and sorted. Alternatively,
>>>> we could indicate to ginExtractEntries() if the entries are already
>>>> deduplicated and sorted. If we don't want to alter the signature of the
>>>> extract value function, we could e.g. use the MSB of the nentries
>>>> argument.
>>>
>>> Yeah, this seems wrong as it is. You're assuming that if the extract
>>> function returns nullFlags == NULL, the array is already sorted and
>>> deduped.
>>
>> As said, that was just for demonstration purposes of the possible gains.
>> I've changed the code now such that the extractValue function of the GIN
>> index can indicate via the third argument uniqueAndSorted, if the
>> returned keys are already unique and sorted.
>>
>> Unfortunately, it seems like this patch regresses performance. See
>> measurements below. I haven't had the time to investigate why that is.
>> It's pretty counter intuitive, given that this patch effectively only
>> removes code. Maybe you could re-test patch 0004 and share your runtimes?
>
> Looking at 0001 and 0004 patches and ginExtractEntries(), the way
> ginExtractEntries() handles NULLs looks a little inefficient. It treats
> NULLs as proper entries, passing them through qsort() for deduplication
> and comparing them with cmpEntries(). But the end result is always that
> if the opclass's extractValueFn() function returned any NULLs, then
> there's exctly one GIN_CAT_NULL_KEY as the last entry of the result
> array. Surely we could be smarter about how we accomplish that. Your
> 0004 patch eliminates the deduplication overhead altogether, which is
> great of course, but the point remains for when we still need the
> deduplication.
Good observation. I like this idea. I've focused on pg_trgm but making
this code faster is certainly useful.
Given that doing the sort on pre-sorted input apparently doesn't add
measurable overhead, according to my benchmark results, we can apply
your patch and leave mine out for the moment being.
That's btw. also the reason for why 0002 doesn't show much gain: when
the data is pre-sorted, cmpEntries() is not called as much.
>
> Attached is an attempt at that. It avoids the null-checks in
> cmpEntries(), saving a few cycles. That's drowned in noise with your
> test cases, but with the attached test case with int arrays, I'm seeing
> a 1-2 % gain. That's not much, but I think it's still worth doing
> because it makes the code a little simpler too, IMHO. (I didn't test it
> together with the rest of your patches.)
Performance is death by a thousand cuts and that's definitely the case
for the GIN index code. I'm all for putting in these small improvements
because they'll add up and what's now 2% can be 10% once other
optimization are in.
I took a look at your patch. Overall looks good to me. Just a few comments:
1) You should be able to create the categories array without the need
for the subsequent for loop as follows:
StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
*categories = palloc0_array(GinNullCategory, (nentries + (hasNull ? 1 : 0));
2) Your test case seems sub-optimal to show the gains. The arrays don't
contain any NULL values and are also already sorted. Or I'm missing
something.
3) Could you try your patch in conjunction with 0002 on data that is not
pre-sorted and then check if 0002 has more impact? That way cmpEntries()
should be called much more often.
4) Have you checked if there are regression tests that exercise this
code? If not, how about adding some?
>>> Perhaps we should introduce a new qunique_eq() function with a different
>>> callback signature:
>>>
>>> /* like qunique(), but the callback function returns true/false rather
>>> than int */
>>> static inline size_t
>>> qunique_eq(void *array, size_t elements, size_t width,
>>> bool (*equal) (const void *, const void *))
>>>
>>
>> I would prefer to change qunique() instead. That would enforce using an
>> adequate comparison function from the get go. There are only ~15 calls
>> to qunique(). So refactoring this should also be a fairly small patch. I
>> can do that if there's agreement for that approach.
>
> Works for me.
>
> At quick glance, most if not all of the qunique() calls call qsort()
> just before qunique(). I wonder if we should have a single "sort and
> deduplicate" function, instead. It could perhaps do some deduplication
> "on the go", or other optimizations.
If it's just for deduplication purposes and the data doesn't have to end
up sorted, something based on a hash map should be even faster. How
about we start with changing the qunique() comparator signature and as a
2nd step take a closer look at how to go about providing a function that
does it in one go?
If you agree, I'll share a patch here next week.
>>> Did you measure how big is the impact from each individual patch?
>>> Patches 1 and 2 seem pretty much ready to be committed, but I wonder if
>>> they make any difference on their own.
>>
>> Here is the impact of each patch. I ran again CREATE INDEX three times
>> and took the fastest run. The run of each patch includes all previous
>> patches as well. For example, the timings for patch 0003 were measured
>> with a binary that also had patch 0002 and 0001 applied. To get the
>> impact of each patch in isolation, the delta to the previous run was
>> taken.
>>
>> Code | movies |delta | lineitem | delta
>> ------------------------------------|--------|-------|------------------
>> master | 10,311 | 0 | 256,986 | 0
>> v1-0001-Inline-ginCompareAttEntries | 9,694 | 617 | 239,778 | 17,208
>> v1-0002-Optimized-comparison-func | 9,510 | 184 | 238,094 | 1,684
>> v1-0003-Use-sort_template.h | 8,661 | 849 | 231,190 | 6,904
>> v1-0004-Avoid-dedup-and-sort-in | 9,305 | -644 | 232,472 | -1,282
>> v1-0005-Make-btint4cmp-branchless | 8,240 | 1,065 | 228,387 | 4,085
>> v1-0006-Use-radix-sort | 6,976 | 1,264 | 207,687 | 20,700
>> v1-0007-Faster-qunique-comparator | 5,911 | 1,065 | 203,744 | 3,943
>> v1-0008-Add-ASCII-fastpath | 3,409 | 2,502 | 161,469 | 42,275
>>
>> Attached is v2 of the patch set with the aforementioned changes. I've
>> also fixed the white space errors in 0003, 0004 and 0008, as reported by
>> Kirill.
>
> Thanks, I pushed patch 0001 now, that's a simple and clear win.
Great. Thanks.
What about the other patches? 0003 and 0007 are also pretty simple and
IMHO uncontroversial while giving decent savings.
For 0006 I would make the code also work for char being unsigned. That's
still missing.
Any thoughts about 0008 and my findings regarding the to lower-case
conversion for ASCII? Adding to pg_locale_struct if it's save to use
ASCII-style tolower should be straight forward and then the code should
be correct.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-12 22:10 Heikki Linnakangas <hlinnaka@iki.fi>
parent: David Geier <geidav.pg@gmail.com>
1 sibling, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-01-12 22:10 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; pgsql-hackers
On 09/01/2026 14:06, David Geier wrote:
> On 06.01.2026 18:00, Heikki Linnakangas wrote:
>> On 05/01/2026 17:01, David Geier wrote:
>>> v1-0008-Add-ASCII-fastpath-to-generate_trgm_only.patch: Typically lots
>>> of text is actually ASCII. Hence, we provide a fast path for this case
>>> which is exercised if the MSB of the current character is unset.
>>
>> This uses pg_ascii_tolower() when for ASCII characters when built with
>> the IGNORECASE. I don't think that's correct, if the proper collation
>> would do something more complicated for than what pg_ascii_tolower() does.
>
> Oh, that's evil. I had tested that specifically. But it only worked
> because the code in master uses str_tolower() with
> DEFAULT_COLLATION_OID. So using a different locale like in the following
> example does something different than when creating a database with the
> same locale.
>
> postgres=# select lower('III' COLLATE "tr_TR");
> lower
> -------
> ııı
>
> postgres=# select show_trgm('III' COLLATE "tr_TR");
> show_trgm
> -------------------------
> {" i"," ii","ii ",iii}
> (1 row)
>
> But when using tr_TR as default locale of the database the following
> happens:
>
> postgres=# select lower('III' COLLATE "tr_TR");
> lower
> -------
> ııı
>
> postgres=# select show_trgm('III');sü
> show_trgm
> ---------------------------------------
> {0xbbd8dd,0xf26fab,0xf31e1a,0x2af4f1}
>
> I'm wondering if that's intentional to begin with. Shouldn't the code
> instead pass PG_GET_COLLATION() to str_tolower()? Might require some
> research to see how other index types handle locales.
>
> Coming back to the original problem: the lengthy comment at the top of
> pg_locale_libc.c, suggests that in some cases ASCII characters are
> handled the pg_ascii_tolower() way for the default locale. See for
> example tolower_libc_mb(). So a character by character conversion using
> that function will yield a different result than strlower_libc_mb(). I'm
> wondering why that is.
Hmm, yeah, that feels funny. The trigram code predates per-column
collation support, so I guess we never really thought through how it
should interact with COLLATE clauses.
> Anyways, we could limit the optimization to only kick in when the used
> locale follows the same rules as pg_ascii_tolower(). We could test that
> when creating the locale and store that info in pg_locale_struct.
I think that's only possible for libc locales, which operate one
character at a time. In ICU locales, lower-casing a character can depend
on the surrounding characters, so you cannot just test the conversion of
every ascii character individually. It would make sense for libc locales
though, and I hope the ICU functions are a little faster anyway.
Although, we probably should be using case-folding rather than
lower-casing with ICU locales anyway. Case-folding is designed for
string matching. It'd be a backwards-compatibility breaking change, though.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-14 12:27 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: David Geier @ 2026-01-14 12:27 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
Hi Heikki!
On 09.01.2026 22:02, David Geier wrote:
> Given that doing the sort on pre-sorted input apparently doesn't add
> measurable overhead, according to my benchmark results, we can apply
> your patch and leave mine out for the moment being.
I've removed my patch from the patch set in favor of your patch. I've
adapted your patch slightly as follows:
- I replaced the custom for-loop by qunique()
- I switched to sort_template.h which gives a nice speedup as it can do
some things more efficiently now where the entries are simple Datums.
- I use palloc0_array() to initialize the array to GIN_CAT_NORM_KEY in
one go.
Your patch together with my changes gives a 20% speedup on a table with
arrays of 1000 elements and 10% NULLs. See attached test script.
> That's btw. also the reason for why 0002 doesn't show much gain: when
> the data is pre-sorted, cmpEntries() is not called as much.
This turned out to be not the case. I tested 0002 with the attached
script but that neither showed any significant improvements. It's still
curious to me because cmpEntries() is called hundreds of millions of
times and the disassembly shows that the optimized function indeed
directly calls the operator rather than having the indirection via
FunctionCall2Coll().
Anyways, I've removed the patch from the patch set for the moment being.
>>> I would prefer to change qunique() instead. That would enforce using an
>>> adequate comparison function from the get go. There are only ~15 calls
>>> to qunique(). So refactoring this should also be a fairly small patch. I
>>> can do that if there's agreement for that approach.
>>
>> Works for me.
>>
>> At quick glance, most if not all of the qunique() calls call qsort()
>> just before qunique(). I wonder if we should have a single "sort and
>> deduplicate" function, instead. It could perhaps do some deduplication
>> "on the go", or other optimizations.
>
> If it's just for deduplication purposes and the data doesn't have to end
> up sorted, something based on a hash map should be even faster. How
> about we start with changing the qunique() comparator signature and as a
> 2nd step take a closer look at how to go about providing a function that
> does it in one go?
Thinking about this some more: ideally we have two functions: something
like deduplicateArray() and sortAndDeduplicateArray(). We could
initially implement deduplicateArray() on top of
sortAndDeduplicateArray(). If we ever find a case that needs
optimization, and doesn't require the data to actually end up sorted, we
can implement deduplicateArray() e.g. on top of simplehash.h.
I'll draft a patch and submit it in a separate thread.
> What about the other patches? 0003 and 0007 are also pretty simple and
> IMHO uncontroversial while giving decent savings.
I've reordered the patches such that the ones that I think are
uncontroversial, small and ready to be committed are at the beginning
(patches 0001 - 0004). The radix sort and ASCII fast-path patches come
last (0005 and 0006). I would like to first concentrate on getting 0001
- 0004 in and then get back to 0005 and 0006.
I remeasured the savings of 0001 - 0004, which comes on top of the
already committed patch that inlined the comparison function, which gave
another ~5%:
Data set | Patched (ms) | Master (ms) | Speedup
--------------------|--------------|--------------|----------
movies(plot) | 8,058 | 10,311 | 1.27x
lineitem(l_comment) | 223,233 | 256,986 | 1.19x
--
David Geier
Attachments:
[application/sql] table_with_random_int_arrays.sql (633B, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/2-table_with_random_int_arrays.sql)
download
[text/x-patch] v3-0006-Add-ASCII-fastpath-to-generate_trgm_only.patch (4.0K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/3-v3-0006-Add-ASCII-fastpath-to-generate_trgm_only.patch)
download | inline diff:
From b7deb09e0717ee76d98fd1a40cff4b749bfe1c36 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Fri, 14 Nov 2025 11:37:40 +0100
Subject: [PATCH v3 6/6] Add ASCII fastpath to generate_trgm_only()
---
contrib/pg_trgm/trgm_op.c | 124 ++++++++++++++++++++------------------
1 file changed, 65 insertions(+), 59 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 39b586f5b9a..d2087b3a45e 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,32 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-/*
- * Finds first word in string, returns pointer to the word,
- * endword points to the character after word
- */
-static char *
-find_word(char *str, int lenstr, char **endword, int *charlen)
-{
- char *beginword = str;
-
- while (beginword - str < lenstr && !ISWORDCHR(beginword))
- beginword += pg_mblen(beginword);
-
- if (beginword - str >= lenstr)
- return NULL;
-
- *endword = beginword;
- *charlen = 0;
- while (*endword - str < lenstr && ISWORDCHR(*endword))
- {
- *endword += pg_mblen(*endword);
- (*charlen)++;
- }
-
- return beginword;
-}
-
/*
* Reduce a trigram (three possibly multi-byte characters) to a trgm,
* which is always exactly three bytes. If we have three single-byte
@@ -337,58 +311,90 @@ make_trigrams(trgm *tptr, char *str, int bytelen, int charlen)
static int
generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
{
- trgm *tptr;
- char *buf;
- int charlen,
- bytelen;
- char *bword,
- *eword;
+ trgm *tptr = trg;
+ char *buf;
if (slen + LPADDING + RPADDING < 3 || slen == 0)
return 0;
- tptr = trg;
-
- /* Allocate a buffer for case-folded, blank-padded words */
- buf = (char *) palloc(slen * pg_database_encoding_max_length() + 4);
+ buf = palloc_array(char, slen * pg_database_encoding_max_length() + 4 + 1);
+ memset(buf, ' ', LPADDING);
- if (LPADDING > 0)
+ for (int i = 0; i < slen; )
{
- *buf = ' ';
- if (LPADDING > 1)
- *(buf + 1) = ' ';
- }
+ int num_bytes = LPADDING;
+ int num_chars = LPADDING;
+ char *word;
- eword = str;
- while ((bword = find_word(eword, slen - (eword - str), &eword, &charlen)) != NULL)
- {
+ /* Extract next word */
+ while (i < slen)
+ {
+ if ((str[i] & 0x80) == 0) /* Fast path for ASCII-only */
+ {
+ if (isalnum(str[i]))
+ {
#ifdef IGNORECASE
- bword = str_tolower(bword, eword - bword, DEFAULT_COLLATION_OID);
- bytelen = strlen(bword);
+ buf[num_bytes++] = pg_ascii_tolower(str[i++]);
#else
- bytelen = eword - bword;
+ buf[num_bytes++] = str[i++];
#endif
+ }
+ else
+ {
+ i++;
+ break;
+ }
+ }
+ else
+ {
+ const int mblen = pg_mblen(str + i);
+ Assert(mblen >= 2); /* Otherwise, it would be ASCII */
+
+ if (ISWORDCHR(str + i))
+ {
+ memcpy(buf + num_bytes, str + i, mblen);
+ num_bytes += mblen;
+ i += mblen;
+ }
+ else
+ {
+ i += mblen;
+ break;
+ }
+ }
+
+ num_chars++;
+ }
- memcpy(buf + LPADDING, bword, bytelen);
+ if (num_chars > LPADDING)
+ {
+ memset(buf + num_bytes, ' ', RPADDING);
+ num_bytes += RPADDING;
+ num_chars += RPADDING;
+ word = buf;
#ifdef IGNORECASE
- pfree(bword);
+ if (num_chars != num_bytes)
+ {
+ word = str_tolower(buf, num_bytes, DEFAULT_COLLATION_OID);
+ num_bytes = strlen(word); /* String can get shorter from lower-casing */
+ }
#endif
- buf[LPADDING + bytelen] = ' ';
- buf[LPADDING + bytelen + 1] = ' ';
+ if (bounds)
+ bounds[tptr - trg] |= TRGM_BOUND_LEFT;
+
+ tptr = make_trigrams(tptr, word, num_bytes, num_chars);
+
+ if (bounds)
+ bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
- /* Calculate trigrams marking their bounds if needed */
- if (bounds)
- bounds[tptr - trg] |= TRGM_BOUND_LEFT;
- tptr = make_trigrams(tptr, buf, bytelen + LPADDING + RPADDING,
- charlen + LPADDING + RPADDING);
- if (bounds)
- bounds[tptr - trg - 1] |= TRGM_BOUND_RIGHT;
+ if (word != buf)
+ pfree(word);
+ }
}
pfree(buf);
-
return tptr - trg;
}
--
2.51.0
[text/x-patch] v3-0005-Optimize-generate_trgm-with-radix-sort.patch (2.9K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/4-v3-0005-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From 24b0f4583bf58e3f5b0a7266cacbf38973801f15 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v3 5/6] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 64 +++++++++++++++++++++++++++++++++------
1 file changed, 54 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index d616cf3845f..39b586f5b9a 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -163,14 +163,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
}
/* Define our specialized sort function name */
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
@@ -400,6 +392,58 @@ generate_trgm_only(trgm *trg, char *str, int slen, TrgmBound *bounds)
return tptr - trg;
}
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ Assert(to == buffer);
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
+
/*
* Guard against possible overflow in the palloc requests below. (We
* don't worry about the additive constants, since palloc can detect
@@ -446,7 +490,7 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
@@ -998,7 +1042,7 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
--
2.51.0
[text/x-patch] v3-0004-Faster-qunique-comparator-in-generate_trgm.patch (2.0K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/5-v3-0004-Faster-qunique-comparator-in-generate_trgm.patch)
download | inline diff:
From b73f679c39671fe3b01652864dd67d8d6a732327 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 12 Nov 2025 14:27:13 +0100
Subject: [PATCH v3 4/6] Faster qunique() comparator in generate_trgm()
---
contrib/pg_trgm/trgm_op.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 6af120fa1ad..d616cf3845f 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -139,6 +139,14 @@ CMPTRGM_UNSIGNED(const void *a, const void *b)
: CMPPCHAR_UNS(a, b, 2));
}
+static inline int
+CMPTRGM_EQ(const void *a, const void *b)
+{
+ char *aa = (char *)a;
+ char *bb = (char *)b;
+ return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
+}
+
/*
* This gets called on the first call. It replaces the function pointer so
* that subsequent calls are routed directly to the chosen implementation.
@@ -438,15 +446,11 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -994,15 +998,11 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v3-0003-Make-btint4cmp-branchless.patch (1.0K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/6-v3-0003-Make-btint4cmp-branchless.patch)
download | inline diff:
From 29fc859f9a2813c2b23ed18196e7b6f970db63e0 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v3 3/6] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 8425805a292..bf081975e3c 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -60,6 +60,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -202,12 +203,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
[text/x-patch] v3-0002-Optimize-generate_trgm-with-sort_template.h.patch (2.6K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/7-v3-0002-Optimize-generate_trgm-with-sort_template.h.patch)
download | inline diff:
From 4eb42c4ff3869c1c18a883faf4071e98e4fd6eca Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:11 +0100
Subject: [PATCH v3 2/6] Optimize generate_trgm() with sort_template.h
---
contrib/pg_trgm/trgm_op.c | 47 ++++++++++++++++++++++++++++++---------
1 file changed, 37 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 81182a15e07..6af120fa1ad 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -154,6 +154,23 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
+/* Define our specialized sort function name */
+#define ST_SORT trigram_qsort_signed
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
+#define ST_SORT trigram_qsort_unsigned
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
/*
* Deprecated function.
* Use "pg_trgm.similarity_threshold" GUC variable instead of this function.
@@ -209,12 +226,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-static int
-comp_trgm(const void *a, const void *b)
-{
- return CMPTRGM(a, b);
-}
-
/*
* Finds first word in string, returns pointer to the word,
* endword points to the character after word
@@ -426,8 +437,16 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -974,8 +993,16 @@ generate_wildcard_trgm(const char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.51.0
[text/x-patch] v3-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch (7.8K, ../../8a311d7b-9101-4947-a700-6a7821678462@gmail.com/8-v3-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch)
download | inline diff:
From 527e03f05ec4513fcaa59cef7d4c9c3d38ea0e18 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 14 Jan 2026 10:54:32 +0100
Subject: [PATCH v3 1/6] Optimize sort and deduplication in ginExtractEntries
---
src/backend/access/gin/ginutil.c | 152 +++++++++++++------------------
src/include/access/gin_private.h | 2 +-
2 files changed, 66 insertions(+), 88 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d205093e21d..5be5e22487f 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -28,6 +28,7 @@
#include "utils/index_selfuncs.h"
#include "utils/rel.h"
#include "utils/typcache.h"
+#include "lib/qunique.h"
/*
@@ -387,19 +388,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Support for sorting key datums in ginExtractEntries
- *
- * Note: we only have to worry about null and not-null keys here;
- * ginExtractEntries never generates more than one placeholder null,
- * so it doesn't have to sort those.
- */
-typedef struct
-{
- Datum datum;
- bool isnull;
-} keyEntryData;
-
typedef struct
{
FmgrInfo *cmpDatumFunc;
@@ -410,24 +398,14 @@ typedef struct
static int
cmpEntries(const void *a, const void *b, void *arg)
{
- const keyEntryData *aa = (const keyEntryData *) a;
- const keyEntryData *bb = (const keyEntryData *) b;
+ const Datum *aa = (const Datum *) a;
+ const Datum *bb = (const Datum *) b;
cmpEntriesArg *data = (cmpEntriesArg *) arg;
int res;
- if (aa->isnull)
- {
- if (bb->isnull)
- res = 0; /* NULL "=" NULL */
- else
- res = 1; /* NULL ">" not-NULL */
- }
- else if (bb->isnull)
- res = -1; /* not-NULL "<" NULL */
- else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
+ data->collation,
+ *aa, *bb));
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -440,6 +418,14 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
+#define ST_SORT qsort_arg_entries
+#define ST_ELEMENT_TYPE Datum
+#define ST_COMPARE_ARG_TYPE cmpEntriesArg
+#define ST_COMPARE(a, b, arg) cmpEntries(a, b, arg)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
/*
* Extract the index key values from an indexable item
@@ -450,11 +436,13 @@ cmpEntries(const void *a, const void *b, void *arg)
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories)
+ int32 *nentries_p, GinNullCategory **categories_p)
{
Datum *entries;
bool *nullFlags;
- int32 i;
+ GinNullCategory *categories;
+ bool hasNull;
+ int32 nentries;
/*
* We don't call the extractValueFn on a null item. Instead generate a
@@ -462,42 +450,60 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
*/
if (isNull)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_NULL_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_NULL_ITEM;
return entries;
}
/* OK, call the opclass's extractValueFn */
nullFlags = NULL; /* in case extractValue doesn't set it */
+ nentries = 0;
entries = (Datum *)
DatumGetPointer(FunctionCall3Coll(&ginstate->extractValueFn[attnum - 1],
ginstate->supportCollation[attnum - 1],
value,
- PointerGetDatum(nentries),
+ PointerGetDatum(&nentries),
PointerGetDatum(&nullFlags)));
/*
* Generate a placeholder if the item contained no keys.
*/
- if (entries == NULL || *nentries <= 0)
+ if (entries == NULL || nentries <= 0)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_EMPTY_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_EMPTY_ITEM;
return entries;
}
/*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
+ * Scan the items for any NULLs. All NULLs are considered equal, so we
+ * just need to check and remember if there are any. We remove them from
+ * the array here, and if necessary, put back one NULL entry to represent
+ * them all after deduplication.
*/
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
+ hasNull = false;
+ if (nullFlags)
+ {
+ int32 numNonNulls = 0;
+
+ for (int32 i = 0; i < nentries; i++)
+ {
+ if (nullFlags[i])
+ hasNull = true;
+ else
+ {
+ entries[numNonNulls] = entries[i];
+ numNonNulls++;
+ }
+ }
+ nentries = numNonNulls;
+ }
/*
* If there's more than one key, sort and unique-ify.
@@ -506,63 +512,35 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
+ if (nentries > 1)
{
- keyEntryData *keydata;
cmpEntriesArg arg;
-
- keydata = palloc_array(keyEntryData, *nentries);
- for (i = 0; i < *nentries; i++)
- {
- keydata[i].datum = entries[i];
- keydata[i].isnull = nullFlags[i];
- }
-
arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
arg.collation = ginstate->supportCollation[attnum - 1];
arg.haveDups = false;
- qsort_arg(keydata, *nentries, sizeof(keyEntryData),
- cmpEntries, &arg);
- if (arg.haveDups)
- {
- /* there are duplicates, must get rid of 'em */
- int32 j;
-
- entries[0] = keydata[0].datum;
- nullFlags[0] = keydata[0].isnull;
- j = 1;
- for (i = 1; i < *nentries; i++)
- {
- if (cmpEntries(&keydata[i - 1], &keydata[i], &arg) != 0)
- {
- entries[j] = keydata[i].datum;
- nullFlags[j] = keydata[i].isnull;
- j++;
- }
- }
- *nentries = j;
- }
- else
- {
- /* easy, no duplicates */
- for (i = 0; i < *nentries; i++)
- {
- entries[i] = keydata[i].datum;
- nullFlags[i] = keydata[i].isnull;
- }
- }
+ qsort_arg_entries(entries, nentries, &arg);
- pfree(keydata);
+ if (arg.haveDups)
+ nentries = qunique_arg(entries, nentries, sizeof(Datum), cmpEntries, &arg);
}
/*
- * Create GinNullCategory representation from nullFlags.
+ * Create GinNullCategory representation.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
+ categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
+
+ /* Put back a NULL entry, if there were any */
+ if (hasNull)
+ {
+ entries[nentries] = (Datum) 0;
+ categories[nentries] = GIN_CAT_NULL_KEY;
+ nentries++;
+ }
+ *nentries_p = nentries;
+ *categories_p = categories;
return entries;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index e155045ce8a..1e97e6cb3c9 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -99,7 +99,7 @@ extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories);
+ int32 *nentries_p, GinNullCategory **categories_p);
extern OffsetNumber gintuple_get_attrnum(GinState *ginstate, IndexTuple tuple);
extern Datum gintuple_get_key(GinState *ginstate, IndexTuple tuple,
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-21 15:45 David Geier <geidav.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-01-21 15:45 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
>> Oh, that's evil. I had tested that specifically. But it only worked
>> because the code in master uses str_tolower() with
>> DEFAULT_COLLATION_OID. So using a different locale like in the following
>> example does something different than when creating a database with the
>> same locale.
>>
>> postgres=# select lower('III' COLLATE "tr_TR");
>> lower
>> -------
>> ııı
>>
>> postgres=# select show_trgm('III' COLLATE "tr_TR");
>> show_trgm
>> -------------------------
>> {" i"," ii","ii ",iii}
>> (1 row)
>>
>> But when using tr_TR as default locale of the database the following
>> happens:
>>
>> postgres=# select lower('III' COLLATE "tr_TR");
>> lower
>> -------
>> ııı
>>
>> postgres=# select show_trgm('III');sü
>> show_trgm
>> ---------------------------------------
>> {0xbbd8dd,0xf26fab,0xf31e1a,0x2af4f1}
>>
>> I'm wondering if that's intentional to begin with. Shouldn't the code
>> instead pass PG_GET_COLLATION() to str_tolower()? Might require some
>> research to see how other index types handle locales.
>>
>> Coming back to the original problem: the lengthy comment at the top of
>> pg_locale_libc.c, suggests that in some cases ASCII characters are
>> handled the pg_ascii_tolower() way for the default locale. See for
>> example tolower_libc_mb(). So a character by character conversion using
>> that function will yield a different result than strlower_libc_mb(). I'm
>> wondering why that is.
>
> Hmm, yeah, that feels funny. The trigram code predates per-column
> collation support, so I guess we never really thought through how it
> should interact with COLLATE clauses.
I've written a patch to fix that. See [1].
>> Anyways, we could limit the optimization to only kick in when the used
>> locale follows the same rules as pg_ascii_tolower(). We could test that
>> when creating the locale and store that info in pg_locale_struct.
>
> I think that's only possible for libc locales, which operate one
> character at a time. In ICU locales, lower-casing a character can depend
> on the surrounding characters, so you cannot just test the conversion of
> every ascii character individually. It would make sense for libc locales
> though, and I hope the ICU functions are a little faster anyway.
>
> Although, we probably should be using case-folding rather than lower-
> casing with ICU locales anyway. Case-folding is designed for string
> matching. It'd be a backwards-compatibility breaking change, though.
Oh, I wasn't ware of that. Doing it only for libc locales seems still
useful.
Good point with the casefolding. I'll look into that.
How do we usually go about such backwards-compatibility breaking
changes? Could we have pg_upgrade reindex all GIN indexes? Would that be
acceptable?
[1]
https://www.postgresql.org/message-id/flat/db087c3e-230e-4119-8a03-8b5d74956bc2%40gmail.com
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-21 20:50 Matthias van de Meent <boekewurm+postgres@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Matthias van de Meent @ 2026-01-21 20:50 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
On Wed, 21 Jan 2026 at 16:45, David Geier <geidav.pg@gmail.com> wrote:
>
> How do we usually go about such backwards-compatibility breaking
> changes?
When it concerns a bug, we mention the change in the release notes
with a warning to reindex affected indexes to be sure no known
corruption remains. See e.g. the final entry in the PG18 release
notes' migration section here:
https://www.postgresql.org/docs/18/release-18.html#RELEASE-18-MIGRATION.
> Could we have pg_upgrade reindex all GIN indexes? Would that be
> acceptable?
No. We'd handle this like any other collation/opclass fixes; we ask
users to reindex their indexes in their own time after they've
upgraded their cluster. Note that in this case it concerns an issue
with just one GIN opclass, not all GIN indexes; so even if we were to
address this in pg_upgrade it wouldn't be a correct choice to reindex
every GIN index, as only a subset of those would be affected by this
issue.
Generally speaking, pg_upgrade doesn't concern itself with the
validity of the data structures that are described by the catalogs
that it upgrades, it only concerns itself with that it correctly
transcribes the catalogs from one version to another, and that the
data files of the old cluster are transfered correctly without
changes.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-01-23 10:18 David Geier <geidav.pg@gmail.com>
parent: Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-01-23 10:18 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
Hi Matthias,
On 21.01.2026 21:50, Matthias van de Meent wrote:
> On Wed, 21 Jan 2026 at 16:45, David Geier <geidav.pg@gmail.com> wrote:
>>
>> How do we usually go about such backwards-compatibility breaking
>> changes?
>
> When it concerns a bug, we mention the change in the release notes
> with a warning to reindex affected indexes to be sure no known
> corruption remains. See e.g. the final entry in the PG18 release
> notes' migration section here:
> https://www.postgresql.org/docs/18/release-18.html#RELEASE-18-MIGRATION.
>
>> Could we have pg_upgrade reindex all GIN indexes? Would that be
>> acceptable?
>
> No. We'd handle this like any other collation/opclass fixes; we ask
> users to reindex their indexes in their own time after they've
> upgraded their cluster. Note that in this case it concerns an issue
> with just one GIN opclass, not all GIN indexes; so even if we were to
> address this in pg_upgrade it wouldn't be a correct choice to reindex
> every GIN index, as only a subset of those would be affected by this
> issue.
>
> Generally speaking, pg_upgrade doesn't concern itself with the
> validity of the data structures that are described by the catalogs
> that it upgrades, it only concerns itself with that it correctly
> transcribes the catalogs from one version to another, and that the
> data files of the old cluster are transfered correctly without
> changes.
Thanks for the clarifications and the link to the release notes. That's
very helpful. Then I know how to move on and will update the patch
accordingly.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-03-02 12:17 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-03-02 12:17 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
On 23.01.2026 11:18, David Geier wrote:
> Hi Matthias,
>
> On 21.01.2026 21:50, Matthias van de Meent wrote:
>> On Wed, 21 Jan 2026 at 16:45, David Geier <geidav.pg@gmail.com> wrote:
>>>
>>> How do we usually go about such backwards-compatibility breaking
>>> changes?
>>
>> When it concerns a bug, we mention the change in the release notes
>> with a warning to reindex affected indexes to be sure no known
>> corruption remains. See e.g. the final entry in the PG18 release
>> notes' migration section here:
>> https://www.postgresql.org/docs/18/release-18.html#RELEASE-18-MIGRATION.
>>
>>> Could we have pg_upgrade reindex all GIN indexes? Would that be
>>> acceptable?
>>
>> No. We'd handle this like any other collation/opclass fixes; we ask
>> users to reindex their indexes in their own time after they've
>> upgraded their cluster. Note that in this case it concerns an issue
>> with just one GIN opclass, not all GIN indexes; so even if we were to
>> address this in pg_upgrade it wouldn't be a correct choice to reindex
>> every GIN index, as only a subset of those would be affected by this
>> issue.
>>
>> Generally speaking, pg_upgrade doesn't concern itself with the
>> validity of the data structures that are described by the catalogs
>> that it upgrades, it only concerns itself with that it correctly
>> transcribes the catalogs from one version to another, and that the
>> data files of the old cluster are transfered correctly without
>> changes.
>
> Thanks for the clarifications and the link to the release notes. That's
> very helpful. Then I know how to move on and will update the patch
> accordingly.
Attached are the patches rebased on latest master.
I've removed the ASCII fast-path patch 0006 as it turned out to be more
complicated to make work than expected.
I kept the radix sort patch because it gives a decent speedup but I
would like to focus for now on getting patches 0001 - 0004 merged.
They're all simple and, the way I see it, uncontroversial.
I remeasured the savings of 0001 - 0004, which comes on top of the
already committed patch that inlined the comparison function, which gave
another ~5%:
Data set | Patched (ms) | Master (ms) | Speedup
--------------------|--------------|--------------|----------
movies(plot) | 8,058 | 10,311 | 1.27x
lineitem(l_comment) | 223,233 | 256,986 | 1.19x
I've also registered the change at the commit fest, see
https://commitfest.postgresql.org/patch/6418/.
--
David Geier
Attachments:
[text/x-patch] v4-0005-Optimize-generate_trgm-with-radix-sort.patch (2.9K, ../../a90ebbbd-0d77-49c7-b222-3dbffa4e3b14@gmail.com/2-v4-0005-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From cc377266ea8071bb000f0c05d0aca0ab2012cdfe Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v4 5/5] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 64 +++++++++++++++++++++++++++++++++------
1 file changed, 54 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 07daf111729..1603a55022b 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -235,14 +235,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
}
/* Define our specialized sort function name */
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
@@ -564,6 +556,58 @@ generate_trgm_only(growable_trgm_array *dst, char *str, int slen, TrgmBound **bo
pfree(buf);
}
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ Assert(to == buffer);
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
+
/*
* Make array of trigrams with sorting and removing duplicate items.
*
@@ -589,7 +633,7 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
@@ -1124,7 +1168,7 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
--
2.51.0
[text/x-patch] v4-0004-Faster-qunique-comparator-in-generate_trgm.patch (2.0K, ../../a90ebbbd-0d77-49c7-b222-3dbffa4e3b14@gmail.com/3-v4-0004-Faster-qunique-comparator-in-generate_trgm.patch)
download | inline diff:
From 6d3a0f61928958b51989e10d5f85db2deef210cf Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 12 Nov 2025 14:27:13 +0100
Subject: [PATCH v4 4/5] Faster qunique() comparator in generate_trgm()
---
contrib/pg_trgm/trgm_op.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 85df5ef2310..07daf111729 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -211,6 +211,14 @@ CMPTRGM_UNSIGNED(const void *a, const void *b)
: CMPPCHAR_UNS(a, b, 2));
}
+static inline int
+CMPTRGM_EQ(const void *a, const void *b)
+{
+ char *aa = (char *)a;
+ char *bb = (char *)b;
+ return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
+}
+
/*
* This gets called on the first call. It replaces the function pointer so
* that subsequent calls are routed directly to the chosen implementation.
@@ -581,15 +589,11 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1120,15 +1124,11 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
trg->flag = ARRKEY;
--
2.51.0
[text/x-patch] v4-0003-Make-btint4cmp-branchless.patch (1.0K, ../../a90ebbbd-0d77-49c7-b222-3dbffa4e3b14@gmail.com/4-v4-0003-Make-btint4cmp-branchless.patch)
download | inline diff:
From 49ea44e1bc26d3166239b4efd257c5f8c4f136fd Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v4 3/5] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 1d343377e98..ac16e3d993d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -203,12 +204,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
[text/x-patch] v4-0002-Optimize-generate_trgm-with-sort_template.h.patch (2.6K, ../../a90ebbbd-0d77-49c7-b222-3dbffa4e3b14@gmail.com/5-v4-0002-Optimize-generate_trgm-with-sort_template.h.patch)
download | inline diff:
From ba64b2fed648f9b3fef53e244d2665ddf1ac9ea3 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:11 +0100
Subject: [PATCH v4 2/5] Optimize generate_trgm() with sort_template.h
---
contrib/pg_trgm/trgm_op.c | 47 ++++++++++++++++++++++++++++++---------
1 file changed, 37 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index ee89e548d16..85df5ef2310 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,6 +226,23 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
+/* Define our specialized sort function name */
+#define ST_SORT trigram_qsort_signed
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
+#define ST_SORT trigram_qsort_unsigned
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
/*
* Deprecated function.
* Use "pg_trgm.similarity_threshold" GUC variable instead of this function.
@@ -281,12 +298,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-static int
-comp_trgm(const void *a, const void *b)
-{
- return CMPTRGM(a, b);
-}
-
/*
* Finds first word in string, returns pointer to the word,
* endword points to the character after word
@@ -569,8 +580,16 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1100,8 +1119,16 @@ generate_wildcard_trgm(const char *str, int slen)
len = arr.length;
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
trg->flag = ARRKEY;
--
2.51.0
[text/x-patch] v4-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch (7.8K, ../../a90ebbbd-0d77-49c7-b222-3dbffa4e3b14@gmail.com/6-v4-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch)
download | inline diff:
From 983e5543c0802201d8c2c43dbde0a19ae3d5ed1b Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 14 Jan 2026 10:54:32 +0100
Subject: [PATCH v4 1/5] Optimize sort and deduplication in ginExtractEntries
---
src/backend/access/gin/ginutil.c | 152 +++++++++++++------------------
src/include/access/gin_private.h | 2 +-
2 files changed, 66 insertions(+), 88 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..15f4a686795 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -28,6 +28,7 @@
#include "utils/index_selfuncs.h"
#include "utils/rel.h"
#include "utils/typcache.h"
+#include "lib/qunique.h"
/*
@@ -387,19 +388,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Support for sorting key datums in ginExtractEntries
- *
- * Note: we only have to worry about null and not-null keys here;
- * ginExtractEntries never generates more than one placeholder null,
- * so it doesn't have to sort those.
- */
-typedef struct
-{
- Datum datum;
- bool isnull;
-} keyEntryData;
-
typedef struct
{
FmgrInfo *cmpDatumFunc;
@@ -410,24 +398,14 @@ typedef struct
static int
cmpEntries(const void *a, const void *b, void *arg)
{
- const keyEntryData *aa = (const keyEntryData *) a;
- const keyEntryData *bb = (const keyEntryData *) b;
+ const Datum *aa = (const Datum *) a;
+ const Datum *bb = (const Datum *) b;
cmpEntriesArg *data = (cmpEntriesArg *) arg;
int res;
- if (aa->isnull)
- {
- if (bb->isnull)
- res = 0; /* NULL "=" NULL */
- else
- res = 1; /* NULL ">" not-NULL */
- }
- else if (bb->isnull)
- res = -1; /* not-NULL "<" NULL */
- else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
+ data->collation,
+ *aa, *bb));
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -440,6 +418,14 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
+#define ST_SORT qsort_arg_entries
+#define ST_ELEMENT_TYPE Datum
+#define ST_COMPARE_ARG_TYPE cmpEntriesArg
+#define ST_COMPARE(a, b, arg) cmpEntries(a, b, arg)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
/*
* Extract the index key values from an indexable item
@@ -450,11 +436,13 @@ cmpEntries(const void *a, const void *b, void *arg)
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories)
+ int32 *nentries_p, GinNullCategory **categories_p)
{
Datum *entries;
bool *nullFlags;
- int32 i;
+ GinNullCategory *categories;
+ bool hasNull;
+ int32 nentries;
/*
* We don't call the extractValueFn on a null item. Instead generate a
@@ -462,42 +450,60 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
*/
if (isNull)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_NULL_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_NULL_ITEM;
return entries;
}
/* OK, call the opclass's extractValueFn */
nullFlags = NULL; /* in case extractValue doesn't set it */
+ nentries = 0;
entries = (Datum *)
DatumGetPointer(FunctionCall3Coll(&ginstate->extractValueFn[attnum - 1],
ginstate->supportCollation[attnum - 1],
value,
- PointerGetDatum(nentries),
+ PointerGetDatum(&nentries),
PointerGetDatum(&nullFlags)));
/*
* Generate a placeholder if the item contained no keys.
*/
- if (entries == NULL || *nentries <= 0)
+ if (entries == NULL || nentries <= 0)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_EMPTY_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_EMPTY_ITEM;
return entries;
}
/*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
+ * Scan the items for any NULLs. All NULLs are considered equal, so we
+ * just need to check and remember if there are any. We remove them from
+ * the array here, and if necessary, put back one NULL entry to represent
+ * them all after deduplication.
*/
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
+ hasNull = false;
+ if (nullFlags)
+ {
+ int32 numNonNulls = 0;
+
+ for (int32 i = 0; i < nentries; i++)
+ {
+ if (nullFlags[i])
+ hasNull = true;
+ else
+ {
+ entries[numNonNulls] = entries[i];
+ numNonNulls++;
+ }
+ }
+ nentries = numNonNulls;
+ }
/*
* If there's more than one key, sort and unique-ify.
@@ -506,63 +512,35 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
+ if (nentries > 1)
{
- keyEntryData *keydata;
cmpEntriesArg arg;
-
- keydata = palloc_array(keyEntryData, *nentries);
- for (i = 0; i < *nentries; i++)
- {
- keydata[i].datum = entries[i];
- keydata[i].isnull = nullFlags[i];
- }
-
arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
arg.collation = ginstate->supportCollation[attnum - 1];
arg.haveDups = false;
- qsort_arg(keydata, *nentries, sizeof(keyEntryData),
- cmpEntries, &arg);
- if (arg.haveDups)
- {
- /* there are duplicates, must get rid of 'em */
- int32 j;
-
- entries[0] = keydata[0].datum;
- nullFlags[0] = keydata[0].isnull;
- j = 1;
- for (i = 1; i < *nentries; i++)
- {
- if (cmpEntries(&keydata[i - 1], &keydata[i], &arg) != 0)
- {
- entries[j] = keydata[i].datum;
- nullFlags[j] = keydata[i].isnull;
- j++;
- }
- }
- *nentries = j;
- }
- else
- {
- /* easy, no duplicates */
- for (i = 0; i < *nentries; i++)
- {
- entries[i] = keydata[i].datum;
- nullFlags[i] = keydata[i].isnull;
- }
- }
+ qsort_arg_entries(entries, nentries, &arg);
- pfree(keydata);
+ if (arg.haveDups)
+ nentries = qunique_arg(entries, nentries, sizeof(Datum), cmpEntries, &arg);
}
/*
- * Create GinNullCategory representation from nullFlags.
+ * Create GinNullCategory representation.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
+ categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
+
+ /* Put back a NULL entry, if there were any */
+ if (hasNull)
+ {
+ entries[nentries] = (Datum) 0;
+ categories[nentries] = GIN_CAT_NULL_KEY;
+ nentries++;
+ }
+ *nentries_p = nentries;
+ *categories_p = categories;
return entries;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 7c3b4db94cd..c878546b9d2 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -99,7 +99,7 @@ extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories);
+ int32 *nentries_p, GinNullCategory **categories_p);
extern OffsetNumber gintuple_get_attrnum(GinState *ginstate, IndexTuple tuple);
extern Datum gintuple_get_key(GinState *ginstate, IndexTuple tuple,
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-03-03 17:31 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-03-03 17:31 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
> Attached are the patches rebased on latest master.
>
> I've removed the ASCII fast-path patch 0006 as it turned out to be more
> complicated to make work than expected.
>
> I kept the radix sort patch because it gives a decent speedup but I
> would like to focus for now on getting patches 0001 - 0004 merged.
> They're all simple and, the way I see it, uncontroversial.
>
> I remeasured the savings of 0001 - 0004, which comes on top of the
> already committed patch that inlined the comparison function, which gave
> another ~5%:
>
> Data set | Patched (ms) | Master (ms) | Speedup
> --------------------|--------------|--------------|----------
> movies(plot) | 8,058 | 10,311 | 1.27x
> lineitem(l_comment) | 223,233 | 256,986 | 1.19x
>
> I've also registered the change at the commit fest, see
> https://commitfest.postgresql.org/patch/6418/.
Attached is v5 that removes an incorrect assertion from the radix sort code.
--
David Geier
Attachments:
[text/x-patch] v5-0005-Optimize-generate_trgm-with-radix-sort.patch (2.8K, ../../2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com/2-v5-0005-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From 19988b78b4d3c5baff19277b09a99d08f3289d7e Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v5 5/5] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 63 ++++++++++++++++++++++++++++++++-------
1 file changed, 53 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 07daf111729..d09e319dda1 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -235,14 +235,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
}
/* Define our specialized sort function name */
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
@@ -564,6 +556,57 @@ generate_trgm_only(growable_trgm_array *dst, char *str, int slen, TrgmBound **bo
pfree(buf);
}
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
+
/*
* Make array of trigrams with sorting and removing duplicate items.
*
@@ -589,7 +632,7 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
@@ -1124,7 +1167,7 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ radix_sort_trigrams_signed((trgm *)GETARR(trg), len);
else
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
--
2.51.0
[text/x-patch] v5-0004-Faster-qunique-comparator-in-generate_trgm.patch (2.0K, ../../2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com/3-v5-0004-Faster-qunique-comparator-in-generate_trgm.patch)
download | inline diff:
From 563dd153369a92be1170ba77f0475567c9223f9e Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 12 Nov 2025 14:27:13 +0100
Subject: [PATCH v5 4/5] Faster qunique() comparator in generate_trgm()
---
contrib/pg_trgm/trgm_op.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 85df5ef2310..07daf111729 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -211,6 +211,14 @@ CMPTRGM_UNSIGNED(const void *a, const void *b)
: CMPPCHAR_UNS(a, b, 2));
}
+static inline int
+CMPTRGM_EQ(const void *a, const void *b)
+{
+ char *aa = (char *)a;
+ char *bb = (char *)b;
+ return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
+}
+
/*
* This gets called on the first call. It replaces the function pointer so
* that subsequent calls are routed directly to the chosen implementation.
@@ -581,15 +589,11 @@ generate_trgm(char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1120,15 +1124,11 @@ generate_wildcard_trgm(const char *str, int slen)
if (len > 1)
{
if (GetDefaultCharSignedness())
- {
trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
- }
else
- {
trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
- len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
- }
+
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_EQ);
}
trg->flag = ARRKEY;
--
2.51.0
[text/x-patch] v5-0003-Make-btint4cmp-branchless.patch (1.0K, ../../2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com/4-v5-0003-Make-btint4cmp-branchless.patch)
download | inline diff:
From eefe78c6a12a45eee7f787ea10a4692ec4b28b62 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v5 3/5] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 1d343377e98..ac16e3d993d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -203,12 +204,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
[text/x-patch] v5-0002-Optimize-generate_trgm-with-sort_template.h.patch (2.6K, ../../2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com/5-v5-0002-Optimize-generate_trgm-with-sort_template.h.patch)
download | inline diff:
From 9492ca0439d44d9806f2f0d6128852e48dc105fe Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 13:35:11 +0100
Subject: [PATCH v5 2/5] Optimize generate_trgm() with sort_template.h
---
contrib/pg_trgm/trgm_op.c | 47 ++++++++++++++++++++++++++++++---------
1 file changed, 37 insertions(+), 10 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index ee89e548d16..85df5ef2310 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,6 +226,23 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
+/* Define our specialized sort function name */
+#define ST_SORT trigram_qsort_signed
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
+#define ST_SORT trigram_qsort_unsigned
+#define ST_ELEMENT_TYPE_VOID
+#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
+
/*
* Deprecated function.
* Use "pg_trgm.similarity_threshold" GUC variable instead of this function.
@@ -281,12 +298,6 @@ show_limit(PG_FUNCTION_ARGS)
PG_RETURN_FLOAT4(similarity_threshold);
}
-static int
-comp_trgm(const void *a, const void *b)
-{
- return CMPTRGM(a, b);
-}
-
/*
* Finds first word in string, returns pointer to the word,
* endword points to the character after word
@@ -569,8 +580,16 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1100,8 +1119,16 @@ generate_wildcard_trgm(const char *str, int slen)
len = arr.length;
if (len > 1)
{
- qsort(GETARR(trg), len, sizeof(trgm), comp_trgm);
- len = qunique(GETARR(trg), len, sizeof(trgm), comp_trgm);
+ if (GetDefaultCharSignedness())
+ {
+ trigram_qsort_signed((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_SIGNED);
+ }
+ else
+ {
+ trigram_qsort_unsigned((void *) GETARR(trg), len, sizeof(trgm));
+ len = qunique(GETARR(trg), len, sizeof(trgm), CMPTRGM_UNSIGNED);
+ }
}
trg->flag = ARRKEY;
--
2.51.0
[text/x-patch] v5-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch (7.8K, ../../2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com/6-v5-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch)
download | inline diff:
From 7ab2b2af2d84b1a5e0e79d1ce9dbd4d7b5d4b921 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 14 Jan 2026 10:54:32 +0100
Subject: [PATCH v5 1/5] Optimize sort and deduplication in ginExtractEntries
---
src/backend/access/gin/ginutil.c | 152 +++++++++++++------------------
src/include/access/gin_private.h | 2 +-
2 files changed, 66 insertions(+), 88 deletions(-)
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..15f4a686795 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -28,6 +28,7 @@
#include "utils/index_selfuncs.h"
#include "utils/rel.h"
#include "utils/typcache.h"
+#include "lib/qunique.h"
/*
@@ -387,19 +388,6 @@ GinInitMetabuffer(Buffer b)
((char *) metadata + sizeof(GinMetaPageData)) - (char *) page;
}
-/*
- * Support for sorting key datums in ginExtractEntries
- *
- * Note: we only have to worry about null and not-null keys here;
- * ginExtractEntries never generates more than one placeholder null,
- * so it doesn't have to sort those.
- */
-typedef struct
-{
- Datum datum;
- bool isnull;
-} keyEntryData;
-
typedef struct
{
FmgrInfo *cmpDatumFunc;
@@ -410,24 +398,14 @@ typedef struct
static int
cmpEntries(const void *a, const void *b, void *arg)
{
- const keyEntryData *aa = (const keyEntryData *) a;
- const keyEntryData *bb = (const keyEntryData *) b;
+ const Datum *aa = (const Datum *) a;
+ const Datum *bb = (const Datum *) b;
cmpEntriesArg *data = (cmpEntriesArg *) arg;
int res;
- if (aa->isnull)
- {
- if (bb->isnull)
- res = 0; /* NULL "=" NULL */
- else
- res = 1; /* NULL ">" not-NULL */
- }
- else if (bb->isnull)
- res = -1; /* not-NULL "<" NULL */
- else
- res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
- data->collation,
- aa->datum, bb->datum));
+ res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
+ data->collation,
+ *aa, *bb));
/*
* Detect if we have any duplicates. If there are equal keys, qsort must
@@ -440,6 +418,14 @@ cmpEntries(const void *a, const void *b, void *arg)
return res;
}
+#define ST_SORT qsort_arg_entries
+#define ST_ELEMENT_TYPE Datum
+#define ST_COMPARE_ARG_TYPE cmpEntriesArg
+#define ST_COMPARE(a, b, arg) cmpEntries(a, b, arg)
+#define ST_SCOPE static
+#define ST_DEFINE
+#define ST_DECLARE
+#include "lib/sort_template.h"
/*
* Extract the index key values from an indexable item
@@ -450,11 +436,13 @@ cmpEntries(const void *a, const void *b, void *arg)
Datum *
ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories)
+ int32 *nentries_p, GinNullCategory **categories_p)
{
Datum *entries;
bool *nullFlags;
- int32 i;
+ GinNullCategory *categories;
+ bool hasNull;
+ int32 nentries;
/*
* We don't call the extractValueFn on a null item. Instead generate a
@@ -462,42 +450,60 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
*/
if (isNull)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_NULL_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_NULL_ITEM;
return entries;
}
/* OK, call the opclass's extractValueFn */
nullFlags = NULL; /* in case extractValue doesn't set it */
+ nentries = 0;
entries = (Datum *)
DatumGetPointer(FunctionCall3Coll(&ginstate->extractValueFn[attnum - 1],
ginstate->supportCollation[attnum - 1],
value,
- PointerGetDatum(nentries),
+ PointerGetDatum(&nentries),
PointerGetDatum(&nullFlags)));
/*
* Generate a placeholder if the item contained no keys.
*/
- if (entries == NULL || *nentries <= 0)
+ if (entries == NULL || nentries <= 0)
{
- *nentries = 1;
+ *nentries_p = 1;
entries = palloc_object(Datum);
entries[0] = (Datum) 0;
- *categories = palloc_object(GinNullCategory);
- (*categories)[0] = GIN_CAT_EMPTY_ITEM;
+ *categories_p = palloc_object(GinNullCategory);
+ (*categories_p)[0] = GIN_CAT_EMPTY_ITEM;
return entries;
}
/*
- * If the extractValueFn didn't create a nullFlags array, create one,
- * assuming that everything's non-null.
+ * Scan the items for any NULLs. All NULLs are considered equal, so we
+ * just need to check and remember if there are any. We remove them from
+ * the array here, and if necessary, put back one NULL entry to represent
+ * them all after deduplication.
*/
- if (nullFlags == NULL)
- nullFlags = (bool *) palloc0(*nentries * sizeof(bool));
+ hasNull = false;
+ if (nullFlags)
+ {
+ int32 numNonNulls = 0;
+
+ for (int32 i = 0; i < nentries; i++)
+ {
+ if (nullFlags[i])
+ hasNull = true;
+ else
+ {
+ entries[numNonNulls] = entries[i];
+ numNonNulls++;
+ }
+ }
+ nentries = numNonNulls;
+ }
/*
* If there's more than one key, sort and unique-ify.
@@ -506,63 +512,35 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
* pretty bad too. For small numbers of keys it'd likely be better to use
* a simple insertion sort.
*/
- if (*nentries > 1)
+ if (nentries > 1)
{
- keyEntryData *keydata;
cmpEntriesArg arg;
-
- keydata = palloc_array(keyEntryData, *nentries);
- for (i = 0; i < *nentries; i++)
- {
- keydata[i].datum = entries[i];
- keydata[i].isnull = nullFlags[i];
- }
-
arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
arg.collation = ginstate->supportCollation[attnum - 1];
arg.haveDups = false;
- qsort_arg(keydata, *nentries, sizeof(keyEntryData),
- cmpEntries, &arg);
- if (arg.haveDups)
- {
- /* there are duplicates, must get rid of 'em */
- int32 j;
-
- entries[0] = keydata[0].datum;
- nullFlags[0] = keydata[0].isnull;
- j = 1;
- for (i = 1; i < *nentries; i++)
- {
- if (cmpEntries(&keydata[i - 1], &keydata[i], &arg) != 0)
- {
- entries[j] = keydata[i].datum;
- nullFlags[j] = keydata[i].isnull;
- j++;
- }
- }
- *nentries = j;
- }
- else
- {
- /* easy, no duplicates */
- for (i = 0; i < *nentries; i++)
- {
- entries[i] = keydata[i].datum;
- nullFlags[i] = keydata[i].isnull;
- }
- }
+ qsort_arg_entries(entries, nentries, &arg);
- pfree(keydata);
+ if (arg.haveDups)
+ nentries = qunique_arg(entries, nentries, sizeof(Datum), cmpEntries, &arg);
}
/*
- * Create GinNullCategory representation from nullFlags.
+ * Create GinNullCategory representation.
*/
- *categories = (GinNullCategory *) palloc0(*nentries * sizeof(GinNullCategory));
- for (i = 0; i < *nentries; i++)
- (*categories)[i] = (nullFlags[i] ? GIN_CAT_NULL_KEY : GIN_CAT_NORM_KEY);
+ StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
+ categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
+
+ /* Put back a NULL entry, if there were any */
+ if (hasNull)
+ {
+ entries[nentries] = (Datum) 0;
+ categories[nentries] = GIN_CAT_NULL_KEY;
+ nentries++;
+ }
+ *nentries_p = nentries;
+ *categories_p = categories;
return entries;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 7c3b4db94cd..c878546b9d2 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -99,7 +99,7 @@ extern void GinInitPage(Page page, uint32 f, Size pageSize);
extern void GinInitMetabuffer(Buffer b);
extern Datum *ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
Datum value, bool isNull,
- int32 *nentries, GinNullCategory **categories);
+ int32 *nentries_p, GinNullCategory **categories_p);
extern OffsetNumber gintuple_get_attrnum(GinState *ginstate, IndexTuple tuple);
extern Datum gintuple_get_key(GinState *ginstate, IndexTuple tuple,
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-07 11:27 Heikki Linnakangas <hlinnaka@iki.fi>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 4 replies; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-07 11:27 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: pgsql-hackers
On 03/03/2026 19:31, David Geier wrote:
>> Attached are the patches rebased on latest master.
>>
>> I've removed the ASCII fast-path patch 0006 as it turned out to be more
>> complicated to make work than expected.
>>
>> I kept the radix sort patch because it gives a decent speedup but I
>> would like to focus for now on getting patches 0001 - 0004 merged.
>> They're all simple and, the way I see it, uncontroversial.
>>
>> I remeasured the savings of 0001 - 0004, which comes on top of the
>> already committed patch that inlined the comparison function, which gave
>> another ~5%:
>>
>> Data set | Patched (ms) | Master (ms) | Speedup
>> --------------------|--------------|--------------|----------
>> movies(plot) | 8,058 | 10,311 | 1.27x
>> lineitem(l_comment) | 223,233 | 256,986 | 1.19x
>>
>> I've also registered the change at the commit fest, see
>> https://commitfest.postgresql.org/patch/6418/.
>
> Attached is v5 that removes an incorrect assertion from the radix sort code.
>
> v5-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch
> v5-0002-Optimize-generate_trgm-with-sort_template.h.patch
> v5-0003-Make-btint4cmp-branchless.patch
> v5-0004-Faster-qunique-comparator-in-generate_trgm.patch
> v5-0005-Optimize-generate_trgm-with-radix-sort.patch
Pushed 0001 as commit 6f5ad00ab7.
I squashed 0002 and 0004 into one commit, and did some more refactoring:
I created a trigram_qsort() helper function that calls the signed or
unsigned variant, so that that logic doesn't need to be duplicated in
the callers. For symmetry, I also added a trigram_qunique() helper
function which just calls qunique() with the new, faster CMPTRGM_EQ
comparator. Pushed these as commit 9f3755ea07.
Patch 0003 gives me pause. It's a tiny patch:
> @@ -203,12 +204,7 @@ btint4cmp(PG_FUNCTION_ARGS)
> int32 a = PG_GETARG_INT32(0);
> int32 b = PG_GETARG_INT32(1);
>
> - if (a > b)
> - PG_RETURN_INT32(A_GREATER_THAN_B);
> - else if (a == b)
> - PG_RETURN_INT32(0);
> - else
> - PG_RETURN_INT32(A_LESS_THAN_B);
> + PG_RETURN_INT32(pg_cmp_s32(a, b));
> }
But the comments on the pg_cmp functions say:
> * NB: If the comparator function is inlined, some compilers may produce
> * worse code with these helper functions than with code with the
> * following form:
> *
> * if (a < b)
> * return -1;
> * if (a > b)
> * return 1;
> * return 0;
> *
So, uh, is that really a universal improvement? Is that comment about
producing worse code outdated?
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-08 02:15 John Naylor <johncnaylorls@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
3 siblings, 1 reply; 51+ messages in thread
From: John Naylor @ 2026-04-08 02:15 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On Tue, Apr 7, 2026 at 6:27 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> But the comments on the pg_cmp functions say:
>
> > * NB: If the comparator function is inlined, some compilers may produce
> > * worse code with these helper functions than with code with the
> > * following form:
> > *
> > * if (a < b)
> > * return -1;
> > * if (a > b)
> > * return 1;
> > * return 0;
> > *
>
> So, uh, is that really a universal improvement? Is that comment about
> producing worse code outdated?
No, it's quite recent:
https://www.postgresql.org/message-id/20240212230423.GA3519%40nathanxps13
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-09 11:28 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
3 siblings, 1 reply; 51+ messages in thread
From: Bertrand Drouvot @ 2026-04-09 11:28 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi,
On Tue, Apr 07, 2026 at 02:27:40PM +0300, Heikki Linnakangas wrote:
> On 03/03/2026 19:31, David Geier wrote:
> > > Attached are the patches rebased on latest master.
> > >
> > > I've removed the ASCII fast-path patch 0006 as it turned out to be more
> > > complicated to make work than expected.
> > >
> > > I kept the radix sort patch because it gives a decent speedup but I
> > > would like to focus for now on getting patches 0001 - 0004 merged.
> > > They're all simple and, the way I see it, uncontroversial.
> > >
> > > I remeasured the savings of 0001 - 0004, which comes on top of the
> > > already committed patch that inlined the comparison function, which gave
> > > another ~5%:
> > >
> > > Data set | Patched (ms) | Master (ms) | Speedup
> > > --------------------|--------------|--------------|----------
> > > movies(plot) | 8,058 | 10,311 | 1.27x
> > > lineitem(l_comment) | 223,233 | 256,986 | 1.19x
> > >
> > > I've also registered the change at the commit fest, see
> > > https://commitfest.postgresql.org/patch/6418/.
> >
> > Attached is v5 that removes an incorrect assertion from the radix sort code.
> >
> > v5-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch
> > v5-0002-Optimize-generate_trgm-with-sort_template.h.patch
> > v5-0003-Make-btint4cmp-branchless.patch
> > v5-0004-Faster-qunique-comparator-in-generate_trgm.patch
> > v5-0005-Optimize-generate_trgm-with-radix-sort.patch
>
> Pushed 0001 as commit 6f5ad00ab7.
This commit makes use of StaticAssertStmt() that has been deprecated in
d50c86e74375. The attached, fixes it.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
[text/x-diff] v1-0001-gin-change-remaining-StaticAssertStmt-to-StaticAs.patch (1.4K, ../../adeNWH5pDawDvvR2@ip-10-97-1-34.eu-west-3.compute.internal/2-v1-0001-gin-change-remaining-StaticAssertStmt-to-StaticAs.patch)
download | inline diff:
From a2289b5c7db807592e470b525ac71cfea2a7cba3 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Thu, 9 Apr 2026 11:10:33 +0000
Subject: [PATCH v1] gin: change remaining StaticAssertStmt() to
StaticAssertDecl()
d50c86e74375 added a comment mentioning that StaticAssertStmt is deprecated
but 6f5ad00ab763 made use of it.
Fixing by replacing the StaticAssertStmt() by StaticAssertDecl() at file scope.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
---
src/backend/access/gin/ginutil.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
100.0% src/backend/access/gin/
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d3351fbe8a3..45d1a8fac9f 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -30,6 +30,8 @@
#include "utils/typcache.h"
#include "lib/qunique.h"
+/* GIN_CAT_NORM_KEY must be equal to 0 */
+StaticAssertDecl(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
/*
* GIN handler function: return IndexAmRoutine with access method parameters
@@ -534,7 +536,6 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
/*
* Create GinNullCategory representation.
*/
- StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
/* Put back a NULL entry, if there were any */
--
2.34.1
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-12 18:05 Tom Lane <tgl@sss.pgh.pa.us>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
3 siblings, 2 replies; 51+ messages in thread
From: Tom Lane @ 2026-04-12 18:05 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Heikki Linnakangas <hlinnaka@iki.fi> writes:
> Pushed 0001 as commit 6f5ad00ab7.
This commit has caused Coverity to start complaining that
most of ginExtractEntries() is unreachable:
*** CID 1691468: Control flow issues (DEADCODE)
/srv/coverity/git/pgsql-git/postgresql/src/backend/access/gin/ginutil.c: 495 in ginExtractEntries()
489 /*
490 * Scan the items for any NULLs. All NULLs are considered equal, so we
491 * just need to check and remember if there are any. We remove them from
492 * the array here, and after deduplication, put back one NULL entry to
493 * represent them all.
494 */
>>> CID 1691468: Control flow issues (DEADCODE)
>>> Execution cannot reach this statement: "hasNull = false;".
495 hasNull = false;
496 if (nullFlags)
497 {
498 int32 numNonNulls = 0;
499
500 for (int32 i = 0; i < nentries; i++)
Evidently, it does not realize that the extractValueFn() can change
nentries from its initial value of zero. I wouldn't be too surprised
if that's related to our casting of the pointer to uintptr_t --- that
may cause it to not see the passed pointer as a potential reference
mechanism.
I would just write that off as Coverity not being smart enough, except
that I'm worried that some compiler might make a similar deduction and
break the function completely. Was the switch to a local variable
for nentries really a useful win performance-wise?
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 09:41 Peter Eisentraut <peter@eisentraut.org>
parent: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Peter Eisentraut @ 2026-04-13 09:41 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 09.04.26 13:28, Bertrand Drouvot wrote:
> Hi,
>
> On Tue, Apr 07, 2026 at 02:27:40PM +0300, Heikki Linnakangas wrote:
>> On 03/03/2026 19:31, David Geier wrote:
>>>> Attached are the patches rebased on latest master.
>>>>
>>>> I've removed the ASCII fast-path patch 0006 as it turned out to be more
>>>> complicated to make work than expected.
>>>>
>>>> I kept the radix sort patch because it gives a decent speedup but I
>>>> would like to focus for now on getting patches 0001 - 0004 merged.
>>>> They're all simple and, the way I see it, uncontroversial.
>>>>
>>>> I remeasured the savings of 0001 - 0004, which comes on top of the
>>>> already committed patch that inlined the comparison function, which gave
>>>> another ~5%:
>>>>
>>>> Data set | Patched (ms) | Master (ms) | Speedup
>>>> --------------------|--------------|--------------|----------
>>>> movies(plot) | 8,058 | 10,311 | 1.27x
>>>> lineitem(l_comment) | 223,233 | 256,986 | 1.19x
>>>>
>>>> I've also registered the change at the commit fest, see
>>>> https://commitfest.postgresql.org/patch/6418/.
>>>
>>> Attached is v5 that removes an incorrect assertion from the radix sort code.
>>>
>>> v5-0001-Optimize-sort-and-deduplication-in-ginExtractEntr.patch
>>> v5-0002-Optimize-generate_trgm-with-sort_template.h.patch
>>> v5-0003-Make-btint4cmp-branchless.patch
>>> v5-0004-Faster-qunique-comparator-in-generate_trgm.patch
>>> v5-0005-Optimize-generate_trgm-with-radix-sort.patch
>>
>> Pushed 0001 as commit 6f5ad00ab7.
>
> This commit makes use of StaticAssertStmt() that has been deprecated in
> d50c86e74375. The attached, fixes it.
I think the position of the static assertion is correct, because it
refers to the palloc0_array() that follows. Maybe the comment could be
a bit clearer, like "using palloc0_array requires GIN_CAT_NORM_KEY==0"?
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 11:04 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 51+ messages in thread
From: Bertrand Drouvot @ 2026-04-13 11:04 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi,
On Mon, Apr 13, 2026 at 11:41:02AM +0200, Peter Eisentraut wrote:
> On 09.04.26 13:28, Bertrand Drouvot wrote:
> >
> > This commit makes use of StaticAssertStmt() that has been deprecated in
> > d50c86e74375. The attached, fixes it.
>
> I think the position of the static assertion is correct, because it refers
> to the palloc0_array() that follows. Maybe the comment could be a bit
> clearer, like "using palloc0_array requires GIN_CAT_NORM_KEY==0"?
Yeah that looks better to not lose the connection with palloc0_array() here.
Done that way in the attached and adding new braces to avoid warning from
-Wdeclaration-after-statement.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
[text/x-diff] v2-0001-gin-change-remaining-StaticAssertStmt-to-StaticAs.patch (1.6K, ../../adzNqZdHrWHYCinj@ip-10-97-1-34.eu-west-3.compute.internal/2-v2-0001-gin-change-remaining-StaticAssertStmt-to-StaticAs.patch)
download | inline diff:
From ebf2368d0620461c741d5fe5432f9856d0317848 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Mon, 13 Apr 2026 10:29:11 +0000
Subject: [PATCH v2] gin: change remaining StaticAssertStmt() to
StaticAssertDecl()
d50c86e74375 added a comment mentioning that StaticAssertStmt is deprecated
but 6f5ad00ab763 made use of it.
Fixing by replacing the StaticAssertStmt() by StaticAssertDecl(). Adding
a comment to make it clear that this is connected to palloc0_array().
Add new braces to avoid warning from -Wdeclaration-after-statement.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/2a2f9cb0-f00d-413c-8517-4a3ad220d104%40eisentraut.org
---
src/backend/access/gin/ginutil.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
100.0% src/backend/access/gin/
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index d3351fbe8a3..76d162075a9 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -534,8 +534,11 @@ ginExtractEntries(GinState *ginstate, OffsetNumber attnum,
/*
* Create GinNullCategory representation.
*/
- StaticAssertStmt(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
- categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
+ {
+ /* Using palloc0_array requires GIN_CAT_NORM_KEY==0 */
+ StaticAssertDecl(GIN_CAT_NORM_KEY == 0, "Assuming GIN_CAT_NORM_KEY=0");
+ categories = palloc0_array(GinNullCategory, nentries + (hasNull ? 1 : 0));
+ }
/* Put back a NULL entry, if there were any */
if (hasNull)
--
2.34.1
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 15:03 David Geier <geidav.pg@gmail.com>
parent: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-13 15:03 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; Peter Eisentraut <peter@eisentraut.org>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi!
On 13.04.2026 13:04, Bertrand Drouvot wrote:
> Hi,
>
> On Mon, Apr 13, 2026 at 11:41:02AM +0200, Peter Eisentraut wrote:
>> On 09.04.26 13:28, Bertrand Drouvot wrote:
>>>
>>> This commit makes use of StaticAssertStmt() that has been deprecated in
>>> d50c86e74375. The attached, fixes it.
I cannot find a comment close to StaticAssertStmt() that says it got
deprecated. Is the goal to completely get rid of StaticAssertStmt()?
>> I think the position of the static assertion is correct, because it refers
>> to the palloc0_array() that follows. Maybe the comment could be a bit
>> clearer, like "using palloc0_array requires GIN_CAT_NORM_KEY==0"?
>
> Yeah that looks better to not lose the connection with palloc0_array() here.
> Done that way in the attached and adding new braces to avoid warning from
> -Wdeclaration-after-statement.
Looks good to me.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 15:05 David Geier <geidav.pg@gmail.com>
parent: John Naylor <johncnaylorls@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-13 15:05 UTC (permalink / raw)
To: John Naylor <johncnaylorls@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 08.04.2026 04:15, John Naylor wrote:
> On Tue, Apr 7, 2026 at 6:27 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> But the comments on the pg_cmp functions say:
>>
>>> * NB: If the comparator function is inlined, some compilers may produce
>>> * worse code with these helper functions than with code with the
>>> * following form:
>>> *
>>> * if (a < b)
>>> * return -1;
>>> * if (a > b)
>>> * return 1;
>>> * return 0;
>>> *
>>
>> So, uh, is that really a universal improvement? Is that comment about
>> producing worse code outdated?
Well spotted. Thanks!
>
> No, it's quite recent:
>
> https://www.postgresql.org/message-id/20240212230423.GA3519%40nathanxps13
In my original benchmarks it was faster. I'll rebase the remaining
commits and do some more analysis.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 15:06 David Geier <geidav.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
3 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-13 15:06 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: pgsql-hackers
Hi Heikki!
> Pushed 0001 as commit 6f5ad00ab7.
>
> I squashed 0002 and 0004 into one commit, and did some more refactoring:
> I created a trigram_qsort() helper function that calls the signed or
> unsigned variant, so that that logic doesn't need to be duplicated in
> the callers. For symmetry, I also added a trigram_qunique() helper
> function which just calls qunique() with the new, faster CMPTRGM_EQ
> comparator. Pushed these as commit 9f3755ea07.
Thanks for committing these patches.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 15:22 David Geier <geidav.pg@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 51+ messages in thread
From: David Geier @ 2026-04-13 15:22 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 12.04.2026 20:05, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> Pushed 0001 as commit 6f5ad00ab7.
>
> This commit has caused Coverity to start complaining that
> most of ginExtractEntries() is unreachable:
>
> *** CID 1691468: Control flow issues (DEADCODE)
> /srv/coverity/git/pgsql-git/postgresql/src/backend/access/gin/ginutil.c: 495 in ginExtractEntries()
> 489 /*
> 490 * Scan the items for any NULLs. All NULLs are considered equal, so we
> 491 * just need to check and remember if there are any. We remove them from
> 492 * the array here, and after deduplication, put back one NULL entry to
> 493 * represent them all.
> 494 */
>>>> CID 1691468: Control flow issues (DEADCODE)
>>>> Execution cannot reach this statement: "hasNull = false;".
> 495 hasNull = false;
> 496 if (nullFlags)
> 497 {
> 498 int32 numNonNulls = 0;
> 499
> 500 for (int32 i = 0; i < nentries; i++)
>
> Evidently, it does not realize that the extractValueFn() can change
> nentries from its initial value of zero. I wouldn't be too surprised
> if that's related to our casting of the pointer to uintptr_t --- that
> may cause it to not see the passed pointer as a potential reference
> mechanism.
Curious that we don't see that more frequently for other functions that
have output arguments. But maybe there are just too few?
> I would just write that off as Coverity not being smart enough, except
> that I'm worried that some compiler might make a similar deduction and
> break the function completely. Was the switch to a local variable
> for nentries really a useful win performance-wise?
I haven't benchmarked the variant with using the pointer directly. I can
do that.
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 16:14 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-13 16:14 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 12/04/2026 21:05, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> Pushed 0001 as commit 6f5ad00ab7.
>
> This commit has caused Coverity to start complaining that
> most of ginExtractEntries() is unreachable:
>
> *** CID 1691468: Control flow issues (DEADCODE)
> /srv/coverity/git/pgsql-git/postgresql/src/backend/access/gin/ginutil.c: 495 in ginExtractEntries()
> 489 /*
> 490 * Scan the items for any NULLs. All NULLs are considered equal, so we
> 491 * just need to check and remember if there are any. We remove them from
> 492 * the array here, and after deduplication, put back one NULL entry to
> 493 * represent them all.
> 494 */
>>>> CID 1691468: Control flow issues (DEADCODE)
>>>> Execution cannot reach this statement: "hasNull = false;".
> 495 hasNull = false;
> 496 if (nullFlags)
> 497 {
> 498 int32 numNonNulls = 0;
> 499
> 500 for (int32 i = 0; i < nentries; i++)
>
> Evidently, it does not realize that the extractValueFn() can change
> nentries from its initial value of zero. I wouldn't be too surprised
> if that's related to our casting of the pointer to uintptr_t --- that
> may cause it to not see the passed pointer as a potential reference
> mechanism.
>
> I would just write that off as Coverity not being smart enough, except
> that I'm worried that some compiler might make a similar deduction and
> break the function completely. Was the switch to a local variable
> for nentries really a useful win performance-wise?
I didn't do it for performance, but because I find the function easier
to read that way. We could change it back.
It's a pretty scary thought that a compiler might misoptimize that
though. In the same function we have 'nullFlags', too, as a local
variable, even before this commit. Not sure why Coverity doesn't
complain about that.
> /*
> * PointerGetDatum
> * Returns datum representation for a pointer.
> */
> static inline Datum
> PointerGetDatum(const void *X)
> {
> return (Datum) (uintptr_t) X;
> }
Hmm, is that 'const' incorrect? This function doesn't modify *X, but the
resulting address will be used to modify it. Maybe changing it to
non-const "void *X" would give Coverity a hint.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-13 17:15 Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Bertrand Drouvot @ 2026-04-13 17:15 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Peter Eisentraut <peter@eisentraut.org>; Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi,
On Mon, Apr 13, 2026 at 05:03:11PM +0200, David Geier wrote:
> Hi!
>
> On 13.04.2026 13:04, Bertrand Drouvot wrote:
> > Hi,
> >
> > On Mon, Apr 13, 2026 at 11:41:02AM +0200, Peter Eisentraut wrote:
> >> On 09.04.26 13:28, Bertrand Drouvot wrote:
> >>>
> >>> This commit makes use of StaticAssertStmt() that has been deprecated in
> >>> d50c86e74375. The attached, fixes it.
>
> I cannot find a comment close to StaticAssertStmt() that says it got
> deprecated.
The comment on top of it's definition is:
"
/*
* StaticAssertStmt() was previously used to make static assertions work as a
* statement, but its use is now deprecated.
*/
"
> Is the goal to completely get rid of StaticAssertStmt()?
According to its comment, I'd say so.
> > Yeah that looks better to not lose the connection with palloc0_array() here.
> > Done that way in the attached and adding new braces to avoid warning from
> > -Wdeclaration-after-statement.
>
> Looks good to me.
Thanks for looking at it!
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-14 07:02 David Geier <geidav.pg@gmail.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-14 07:02 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
> I didn't do it for performance, but because I find the function easier
> to read that way. We could change it back.
>
> It's a pretty scary thought that a compiler might misoptimize that
> though. In the same function we have 'nullFlags', too, as a local
> variable, even before this commit. Not sure why Coverity doesn't
> complain about that.
>
>> /*
>> * PointerGetDatum
>> * Returns datum representation for a pointer.
>> */
>> static inline Datum
>> PointerGetDatum(const void *X)
>> {
>> return (Datum) (uintptr_t) X;
>> }
>
> Hmm, is that 'const' incorrect? This function doesn't modify *X, but the
> resulting address will be used to modify it. Maybe changing it to non-
> const "void *X" would give Coverity a hint.
Ah, that could be it.
Is there a way for me to run Coverity on a patch to test that out?
Which Coverity CI do we actually use? Is it this one here [1]?
[1] https://scan.coverity.com/projects/209?
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-14 09:22 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-14 09:22 UTC (permalink / raw)
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>; David Geier <geidav.pg@gmail.com>; +Cc: Peter Eisentraut <peter@eisentraut.org>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 13/04/2026 20:15, Bertrand Drouvot wrote:
> On Mon, Apr 13, 2026 at 05:03:11PM +0200, David Geier wrote:
>> On 13.04.2026 13:04, Bertrand Drouvot wrote:
>>> On Mon, Apr 13, 2026 at 11:41:02AM +0200, Peter Eisentraut wrote:
>>>> On 09.04.26 13:28, Bertrand Drouvot wrote:
>>>>>
>>>>> This commit makes use of StaticAssertStmt() that has been deprecated in
>>>>> d50c86e74375. The attached, fixes it.
>>
>> I cannot find a comment close to StaticAssertStmt() that says it got
>> deprecated.
>
> The comment on top of it's definition is:
>
> "
> /*
> * StaticAssertStmt() was previously used to make static assertions work as a
> * statement, but its use is now deprecated.
> */
> "
>
>> Is the goal to completely get rid of StaticAssertStmt()?
>
> According to its comment, I'd say so.
>
>>> Yeah that looks better to not lose the connection with palloc0_array() here.
>>> Done that way in the attached and adding new braces to avoid warning from
>>> -Wdeclaration-after-statement.
>>
>> Looks good to me.
>
> Thanks for looking at it!
Committed this StaticAssertStmt/Decl() fix, thanks
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-14 13:05 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: David Geier @ 2026-04-14 13:05 UTC (permalink / raw)
To: John Naylor <johncnaylorls@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 13.04.2026 17:05, David Geier wrote:
> On 08.04.2026 04:15, John Naylor wrote:
>> On Tue, Apr 7, 2026 at 6:27 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>>> But the comments on the pg_cmp functions say:
>>>
>>>> * NB: If the comparator function is inlined, some compilers may produce
>>>> * worse code with these helper functions than with code with the
>>>> * following form:
>>>> *
>>>> * if (a < b)
>>>> * return -1;
>>>> * if (a > b)
>>>> * return 1;
>>>> * return 0;
>>>> *
>>>
>>> So, uh, is that really a universal improvement? Is that comment about
>>> producing worse code outdated?
>
> Well spotted. Thanks!
>
>>
>> No, it's quite recent:
>>
>> https://www.postgresql.org/message-id/20240212230423.GA3519%40nathanxps13
FWICS, this would only matter if btint4cmp() would get inlined
somewhere, where the compiler could actually make use of understanding
that parts of the if-cascade are not needed. Andres' example was
return DO_COMPARE(a, b) < 0 ?
(DO_COMPARE(b, c) < 0 ? b : (DO_COMPARE(a, c) < 0 ? c : a))
: (DO_COMPARE(b, c) > 0 ? b : (DO_COMPARE(a, c) < 0 ? a : c));
In the case of btint4cmp(), it's only ever invoked from the function
manager, where it cannot be inlined.
Or are there ways to invoke btint4cmp() that can be inlined, which I'm
unaware of?
> In my original benchmarks it was faster. I'll rebase the remaining
> commits and do some more analysis.
Here is the disassembly and the perf top output of master vs patched. I
compiled with GCC 15.2.0.
The unpatched version of btint4cmp() contains a conditional jump, which
is mispredicted frequently in the sort. The patched version is
completely branchless.
master
======
Dump of assembler code for function btint4cmp:
0x00005aa9e33ccdb0 <+0>: endbr64
0x00005aa9e33ccdb4 <+4>: mov 0x20(%rdi),%edx
0x00005aa9e33ccdb7 <+7>: mov $0x1,%eax
0x00005aa9e33ccdbc <+12>: cmp %edx,0x30(%rdi)
0x00005aa9e33ccdbf <+15>: jl 0x5aa9e33ccdca <btint4cmp+26>
0x00005aa9e33ccdc1 <+17>: setne %al
0x00005aa9e33ccdc4 <+20>: movzbl %al,%eax
0x00005aa9e33ccdc7 <+23>: neg %rax
0x00005aa9e33ccdca <+26>: ret
37.22% pg_trgm.so [.] trigram_qsort_signed.constprop.0
7.99% postgres [.] cmpEntryAccumulator
6.60% postgres [.] ginCombineData
6.03% postgres [.] FunctionCall2Coll
3.19% postgres [.] btint4cmp
2.30% postgres [.] rbt_insert
2.29% pg_trgm.so [.] generate_trgm
2.24% postgres [.] pg_mblen_range
1.77% libc.so.6 [.] __towlower_l
1.73% pg_trgm.so [.] trigram_qsort_signed_med3
1.56% postgres [.] pg_utf2wchar_with_len
Patched
=======
Dump of assembler code for function btint4cmp:
0x000055a69e87bdb0 <+0>: endbr64
0x000055a69e87bdb4 <+4>: mov 0x20(%rdi),%eax
0x000055a69e87bdb7 <+7>: cmp %eax,0x30(%rdi)
0x000055a69e87bdba <+10>: setl %al
0x000055a69e87bdbd <+13>: setg %dl
0x000055a69e87bdc0 <+16>: movzbl %dl,%edx
0x000055a69e87bdc3 <+19>: movzbl %al,%eax
0x000055a69e87bdc6 <+22>: sub %edx,%eax
0x000055a69e87bdc8 <+24>: cltq
0x000055a69e87bdca <+26>: ret
38.07% pg_trgm.so [.] trigram_qsort_signed.constprop.0
7.69% postgres [.] cmpEntryAccumulator
6.96% postgres [.] ginCombineData
3.90% postgres [.] FunctionCall2Coll
2.54% postgres [.] pg_mblen_range
2.40% postgres [.] btint4cmp
2.38% pg_trgm.so [.] generate_trgm
1.86% postgres [.] rbt_insert
1.80% libc.so.6 [.] __towlower_l
1.73% pg_trgm.so [.] trigram_qsort_signed_med3
1.66% postgres [.] pg_utf2wchar_with_len
--
David Geier
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-14 14:24 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-14 14:24 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: pgsql-hackers
On 13.04.2026 17:06, David Geier wrote:
>> I squashed 0002 and 0004 into one commit, and did some more refactoring:
>> I created a trigram_qsort() helper function that calls the signed or
>> unsigned variant, so that that logic doesn't need to be duplicated in
>> the callers. For symmetry, I also added a trigram_qunique() helper
>> function which just calls qunique() with the new, faster CMPTRGM_EQ
>> comparator. Pushed these as commit 9f3755ea07.
>
> Thanks for committing these patches.
Attached are the remaining patches (previously 0003 and 0005) rebased on
latest master. Currently, there's no radix sort variant for the unsigned
char case. Do we care about this case or is it fine if that case runs
slower?
The following perf profiles show that trigram_qsort() goes from ~34%
down to ~7% with the radix sort optimization. The optimized run also
includes the btint4cmp() optimization. Without that the result would be
even better.
With that change we could move on and tackle optimizing
1. 41.52% generate_trgm_only() by e.g. using an ASCII fast-patch
2. 32.72% ginInsertBAEntries() by no longer using the RB-tree but
e.g. also the radix sort
master
- heapam_index_build_range_scan
- 99.40% ginBuildCallback
- ginHeapTupleBulkInsert
- 66.55% ginExtractEntries
- 65.29% FunctionCall3Coll
- gin_extract_value_trgm
- 62.80% generate_trgm
+ 34.33% trigram_qsort (inlined)
+ 26.20% generate_trgm_only
+ 2.23% trigram_qunique (inlined)
+ 1.74% detoast_attr
+ 1.19% qsort_arg_entries
+ 32.72% ginInsertBAEntries
patched
- heapam_index_build_range_scan
- 99.42% ginBuildCallback
- 95.95% ginHeapTupleBulkInsert
- 59.11% ginExtractEntries
- 56.93% FunctionCall3Coll
- gin_extract_value_trgm
- 52.19% generate_trgm
+ 41.52% generate_trgm_only
+ 7.14% trigram_qsort (inlined)
+ 3.53% trigram_qunique (inlined)
+ 4.08% detoast_attr
+ 2.13% qsort_arg_entries
+ 36.78% ginInsertBAEntries
--
David Geier
Attachments:
[text/x-patch] v6-0002-Optimize-generate_trgm-with-radix-sort.patch (2.2K, ../../5650bf75-dcb8-446d-8cba-e626eb44594b@gmail.com/2-v6-0002-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From b20910a814e8c8b64c3d74841fe27a7c12dd927a Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v6 2/2] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 59 +++++++++++++++++++++++++++++++++------
1 file changed, 51 insertions(+), 8 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 0aca9b5826f..f2dbe88ece0 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,13 +226,56 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
@@ -247,7 +290,7 @@ static void
trigram_qsort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ radix_sort_trigrams_signed(array, n);
else
trigram_qsort_unsigned(array, n, sizeof(trgm));
}
--
2.51.0
[text/x-patch] v6-0001-Make-btint4cmp-branchless.patch (1.0K, ../../5650bf75-dcb8-446d-8cba-e626eb44594b@gmail.com/3-v6-0001-Make-btint4cmp-branchless.patch)
download | inline diff:
From 0f39f8618c784299b30566a0bf68beef8b2b8aef Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v6 1/2] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 1d343377e98..ac16e3d993d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -203,12 +204,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-15 11:06 Heikki Linnakangas <hlinnaka@iki.fi>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-15 11:06 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter@eisentraut.org>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 14/04/2026 10:02, David Geier wrote:
>> I didn't do it for performance, but because I find the function easier
>> to read that way. We could change it back.
>>
>> It's a pretty scary thought that a compiler might misoptimize that
>> though. In the same function we have 'nullFlags', too, as a local
>> variable, even before this commit. Not sure why Coverity doesn't
>> complain about that.
>>
>>> /*
>>> * PointerGetDatum
>>> * Returns datum representation for a pointer.
>>> */
>>> static inline Datum
>>> PointerGetDatum(const void *X)
>>> {
>>> return (Datum) (uintptr_t) X;
>>> }
>>
>> Hmm, is that 'const' incorrect? This function doesn't modify *X, but the
>> resulting address will be used to modify it. Maybe changing it to non-
>> const "void *X" would give Coverity a hint.
This was briefly discussed when PointerGetDatum() was changed from a
macro to a static inline function [1]. On that email, Peter pointed out
that the compiler was doing the same deduction that Coverity did now,
i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
function, it cannot change *foo. I'm surprised we dismissed that worry
so quickly. If the compiler optimizes based on that assumption, you can
get incorrect code.
Three alternative fixes were discussed on that thread. Here's a fourth
one that I think is better;
#define PointerGetDatum(X) \
((Datum) (uintptr_t) (true ? (X) : NULL))
I found this trick with the dummy conditional expression at [2]. It
always evaluates to just (X), but it has the effect that you get a
compiler error if (X) is not a pointer.
[1]
https://www.postgresql.org/message-id/812568f2-ff1d-ebd9-aee6-e00d8f2e0fb6%40enterprisedb.com
[2] See "TO_VOID_PTR_EXPR()" at
https://medium.com/@pauljlucas/generic-in-c-d7ab47e3b5ab
> Ah, that could be it.
> Is there a way for me to run Coverity on a patch to test that out?
Not really I'm afraid. I can commit a fix and we'll see if it helps the
next time that Coverity runs (= Sunday).
> Which Coverity CI do we actually use? Is it this one here [1]?
>
> [1] https://scan.coverity.com/projects/209?
Yeah, that's the one, but only the security team has access.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-15 19:12 Peter Eisentraut <peter@eisentraut.org>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: Peter Eisentraut @ 2026-04-15 19:12 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; David Geier <geidav.pg@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 15.04.26 13:06, Heikki Linnakangas wrote:
> On 14/04/2026 10:02, David Geier wrote:
>>> I didn't do it for performance, but because I find the function easier
>>> to read that way. We could change it back.
>>>
>>> It's a pretty scary thought that a compiler might misoptimize that
>>> though. In the same function we have 'nullFlags', too, as a local
>>> variable, even before this commit. Not sure why Coverity doesn't
>>> complain about that.
>>>
>>>> /*
>>>> * PointerGetDatum
>>>> * Returns datum representation for a pointer.
>>>> */
>>>> static inline Datum
>>>> PointerGetDatum(const void *X)
>>>> {
>>>> return (Datum) (uintptr_t) X;
>>>> }
>>>
>>> Hmm, is that 'const' incorrect? This function doesn't modify *X, but the
>>> resulting address will be used to modify it. Maybe changing it to non-
>>> const "void *X" would give Coverity a hint.
>
> This was briefly discussed when PointerGetDatum() was changed from a
> macro to a static inline function [1]. On that email, Peter pointed out
> that the compiler was doing the same deduction that Coverity did now,
> i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
> function, it cannot change *foo. I'm surprised we dismissed that worry
> so quickly. If the compiler optimizes based on that assumption, you can
> get incorrect code.
I don't think this is in evidence. AFAICT, it's just Coverity that is
complaining here, which is its right, but the code is not incorrect.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-15 21:25 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 51+ messages in thread
From: Tom Lane @ 2026-04-15 21:25 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Peter Eisentraut <peter@eisentraut.org> writes:
> On 15.04.26 13:06, Heikki Linnakangas wrote:
>> This was briefly discussed when PointerGetDatum() was changed from a
>> macro to a static inline function [1]. On that email, Peter pointed out
>> that the compiler was doing the same deduction that Coverity did now,
>> i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
>> function, it cannot change *foo. I'm surprised we dismissed that worry
>> so quickly. If the compiler optimizes based on that assumption, you can
>> get incorrect code.
> I don't think this is in evidence. AFAICT, it's just Coverity that is
> complaining here, which is its right, but the code is not incorrect.
Are you sure? This seems like the sort of thing that will bite us on
the rear sometime in the future, as the compiler geeks put in more and
more aggressive optimizations.
I think we should at least test the theory that changing
PointerGetDatum to remove the const cast would silence Coverity's
complaint. If it does not then we're attributing too much
intelligence to Coverity. But if it does, then we've correctly
identified why it's complaining, and we should take seriously the
idea that they aren't the only ones making this sort of deduction
(or won't be for long).
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 08:45 Peter Eisentraut <peter@eisentraut.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 51+ messages in thread
From: Peter Eisentraut @ 2026-04-16 08:45 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 15.04.26 23:25, Tom Lane wrote:
> Peter Eisentraut <peter@eisentraut.org> writes:
>> On 15.04.26 13:06, Heikki Linnakangas wrote:
>>> This was briefly discussed when PointerGetDatum() was changed from a
>>> macro to a static inline function [1]. On that email, Peter pointed out
>>> that the compiler was doing the same deduction that Coverity did now,
>>> i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
>>> function, it cannot change *foo. I'm surprised we dismissed that worry
>>> so quickly. If the compiler optimizes based on that assumption, you can
>>> get incorrect code.
>
>> I don't think this is in evidence. AFAICT, it's just Coverity that is
>> complaining here, which is its right, but the code is not incorrect.
>
> Are you sure? This seems like the sort of thing that will bite us on
> the rear sometime in the future, as the compiler geeks put in more and
> more aggressive optimizations.
>
> I think we should at least test the theory that changing
> PointerGetDatum to remove the const cast would silence Coverity's
> complaint. If it does not then we're attributing too much
> intelligence to Coverity. But if it does, then we've correctly
> identified why it's complaining, and we should take seriously the
> idea that they aren't the only ones making this sort of deduction
> (or won't be for long).
I think it's quite clear to me that Coverity is complaining about this
correctly, in its view of the world. Compilers sometimes complain about
this, too, although in this case they apparently don't look quite as
deeply to do this analysis.
What I'm missing here is, essentially where the previous thread stopped:
What is the overall message that we want to communicate with the API?
If the default assumption is that what pointers converted to Datums
point to should not be modified on the other side (where the Datum is
converted back to a pointer), then the current declaration of
PointerGetDatum() is suitable, and the GIN code can be considered an
exception and we make a special API for that. The previous thread
proposed NonconstPointerGetDatum().
(If this is the resolution, I also have half a patch somewhere that
makes the string input argument for the InputFunctionCall family of
functions const, which also seems intuitively sensible.)
If, on the other hand, the decision is that there is in fact no such
guarantee, that consumers of Datums are free to modify whatever they
seem fit, then we should drop the const of PointerGetDatum and fix the
fallout up the call stack.
The macro proposed by Heikki, I don't know, still doesn't actually
answer this question, just (possibly) makes these warnings go away in a
slightly mysterious way.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 09:49 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-16 09:49 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 16/04/2026 11:45, Peter Eisentraut wrote:
> On 15.04.26 23:25, Tom Lane wrote:
>> Peter Eisentraut <peter@eisentraut.org> writes:
>>> On 15.04.26 13:06, Heikki Linnakangas wrote:
>>>> This was briefly discussed when PointerGetDatum() was changed from a
>>>> macro to a static inline function [1]. On that email, Peter pointed out
>>>> that the compiler was doing the same deduction that Coverity did now,
>>>> i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
>>>> function, it cannot change *foo. I'm surprised we dismissed that worry
>>>> so quickly. If the compiler optimizes based on that assumption, you can
>>>> get incorrect code.
>>
>>> I don't think this is in evidence. AFAICT, it's just Coverity that is
>>> complaining here, which is its right, but the code is not incorrect.
>>
>> Are you sure? This seems like the sort of thing that will bite us on
>> the rear sometime in the future, as the compiler geeks put in more and
>> more aggressive optimizations.
>>
>> I think we should at least test the theory that changing
>> PointerGetDatum to remove the const cast would silence Coverity's
>> complaint. If it does not then we're attributing too much
>> intelligence to Coverity. But if it does, then we've correctly
>> identified why it's complaining, and we should take seriously the
>> idea that they aren't the only ones making this sort of deduction
>> (or won't be for long).
>
> I think it's quite clear to me that Coverity is complaining about this
> correctly, in its view of the world. Compilers sometimes complain about
> this, too, although in this case they apparently don't look quite as
> deeply to do this analysis.
>
> What I'm missing here is, essentially where the previous thread stopped:
> What is the overall message that we want to communicate with the API?
>
> If the default assumption is that what pointers converted to Datums
> point to should not be modified on the other side (where the Datum is
> converted back to a pointer), then the current declaration of
> PointerGetDatum() is suitable, and the GIN code can be considered an
> exception and we make a special API for that. The previous thread
> proposed NonconstPointerGetDatum().
>
> (If this is the resolution, I also have half a patch somewhere that
> makes the string input argument for the InputFunctionCall family of
> functions const, which also seems intuitively sensible.)
>
> If, on the other hand, the decision is that there is in fact no such
> guarantee, that consumers of Datums are free to modify whatever they
> seem fit, then we should drop the const of PointerGetDatum and fix the
> fallout up the call stack.
>
> The macro proposed by Heikki, I don't know, still doesn't actually
> answer this question, just (possibly) makes these warnings go away in a
> slightly mysterious way.
My intention was that if you do:
const foo *ptr;
...
datum = PointerGetDatum(ptr);
Then you're not allowed to modify the contents of *ptr through the
'datum'. But if you do:
foo *ptr;
...
datum = PointerGetDatum(ptr);
Then it is allowed.
I don't know how to tell the compiler exactly that. I tried various
hacks with _Generic(), but couldn't make it work. The macro casts away
the 'const' in the first case, which might disable some optimizations
that the compiler could otherwise do.
We could have all three:
ConstPointerGetDatum(const void *): Promises that the returned Datum is
not used to modify
NonConstPointerGetDatum(void *): No such promise.
PointerGetDatum(): Macro as I proposed. Like NonConstPointerGetDatum(),
but for backwards-compatibility it doesn't emit a warning if you pass a
const pointer to it.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 14:37 Tom Lane <tgl@sss.pgh.pa.us>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: Tom Lane @ 2026-04-16 14:37 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Peter Eisentraut <peter@eisentraut.org>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Heikki Linnakangas <hlinnaka@iki.fi> writes:
> On 16/04/2026 11:45, Peter Eisentraut wrote:
>> What I'm missing here is, essentially where the previous thread stopped:
>> What is the overall message that we want to communicate with the API?
Good point.
>> If the default assumption is that what pointers converted to Datums
>> point to should not be modified on the other side (where the Datum is
>> converted back to a pointer), then the current declaration of
>> PointerGetDatum() is suitable, and the GIN code can be considered an
>> exception and we make a special API for that. The previous thread
>> proposed NonconstPointerGetDatum().
I think there can be no doubt that most functions receiving a
pass-by-ref Datum are not supposed to scribble on the pointed-to
data. So it makes sense to me that PointerGetDatum should carry
an implication of const-ness, and then we need to invent a new
notation to use in the small number of places where that's not
appropriate. I'd capitalize it as NonConstPointerGetDatum,
but other than that nit that naming suggestion seems fine to me.
Of course, then the *real* question is why DatumGetPointer
doesn't deliver a const pointer. But I don't see how to get
there without extremely invasive changes.
> We could have all three:
Not excited about making massive changes for this.
I remain far less certain than Peter is that this discussion has
anything to do with why Coverity is complaining about
ginExtractEntries. I still think we should make some minimum-effort
change to see if the complaint goes away before expending a lot of
brain cells on choosing a final fix.
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 14:43 Andres Freund <andres@anarazel.de>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 0 replies; 51+ messages in thread
From: Andres Freund @ 2026-04-16 14:43 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org, Peter Eisentraut <peter@eisentraut.org>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi,
On April 16, 2026 4:45:55 AM EDT, Peter Eisentraut <peter@eisentraut.org> wrote:
>On 15.04.26 23:25, Tom Lane wrote:
>> Peter Eisentraut <peter@eisentraut.org> writes:
>>> On 15.04.26 13:06, Heikki Linnakangas wrote:
>>>> This was briefly discussed when PointerGetDatum() was changed from a
>>>> macro to a static inline function [1]. On that email, Peter pointed out
>>>> that the compiler was doing the same deduction that Coverity did now,
>>>> i.e. that if you pass the Datum returned by PointerGetDatum(&foo) to a
>>>> function, it cannot change *foo. I'm surprised we dismissed that worry
>>>> so quickly. If the compiler optimizes based on that assumption, you can
>>>> get incorrect code.
>>
>>> I don't think this is in evidence. AFAICT, it's just Coverity that is
>>> complaining here, which is its right, but the code is not incorrect.
>>
>> Are you sure? This seems like the sort of thing that will bite us on
>> the rear sometime in the future, as the compiler geeks put in more and
>> more aggressive optimizations.
>>
>> I think we should at least test the theory that changing
>> PointerGetDatum to remove the const cast would silence Coverity's
>> complaint. If it does not then we're attributing too much
>> intelligence to Coverity. But if it does, then we've correctly
>> identified why it's complaining, and we should take seriously the
>> idea that they aren't the only ones making this sort of deduction
>> (or won't be for long).
>
>I think it's quite clear to me that Coverity is complaining about this correctly, in its view of the world. Compilers sometimes complain about this, too, although in this case they apparently don't look quite as deeply to do this analysis.
>
>What I'm missing here is, essentially where the previous thread stopped: What is the overall message that we want to communicate with the API?
>
>If the default assumption is that what pointers converted to Datums point to should not be modified on the other side (where the Datum is converted back to a pointer), then the current declaration of PointerGetDatum() is suitable, and the GIN code can be considered an exception and we make a special API for that. The previous thread proposed NonconstPointerGetDatum().
To me it seems way way way to dangerous to just redefine what PointerGetDatum() means for all existing callers, without doing an exhaustive verification of all the callers.
Separately from that, it doesn't seem defensible to take a const pointer and return a non const one. Why is that sane?
Greetings,
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 17:30 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-16 17:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter Eisentraut <peter@eisentraut.org>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 16/04/2026 17:37, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> On 16/04/2026 11:45, Peter Eisentraut wrote:
>>> What I'm missing here is, essentially where the previous thread stopped:
>>> What is the overall message that we want to communicate with the API?
>
> Good point.
>
>>> If the default assumption is that what pointers converted to Datums
>>> point to should not be modified on the other side (where the Datum is
>>> converted back to a pointer), then the current declaration of
>>> PointerGetDatum() is suitable, and the GIN code can be considered an
>>> exception and we make a special API for that. The previous thread
>>> proposed NonconstPointerGetDatum().
>
> I think there can be no doubt that most functions receiving a
> pass-by-ref Datum are not supposed to scribble on the pointed-to
> data. So it makes sense to me that PointerGetDatum should carry
> an implication of const-ness, and then we need to invent a new
> notation to use in the small number of places where that's not
> appropriate. I'd capitalize it as NonConstPointerGetDatum,
> but other than that nit that naming suggestion seems fine to me.
That makes sense. My worry is that we're changing the rules in a very
subtle way: It used to be OK to use PointerGetDatum(), pass the
resulting datum to something that modifies it. Now we say it's not OK,
and you must use NonConstPointerGetDatum(). You don't get any compiler
warnings if you use it wrong, except for this one coverity warning
apparently, but it doesn't catch this reliably either.
> Of course, then the *real* question is why DatumGetPointer
> doesn't deliver a const pointer. But I don't see how to get
> there without extremely invasive changes.
Good point.
>> We could have all three:
>
> Not excited about making massive changes for this.
Having all three would be a very localized change in postgres.h.
> I remain far less certain than Peter is that this discussion has
> anything to do with why Coverity is complaining about
> ginExtractEntries. I still think we should make some minimum-effort
> change to see if the complaint goes away before expending a lot of
> brain cells on choosing a final fix.
I think I'm going to commit my proposal to turn PointerGetDatum() back
into a macro, and see if that makes Coverity happy. Then we'll know, and
we can decide on the next steps. Any objections?
One open question is whether we should backpatch any of this. I guess
compilers don't misoptimize this in practice, or we would've gotten more
reports, but I really can't rationalize why not and a new compiler
version might well start hitting this.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-16 17:47 Tom Lane <tgl@sss.pgh.pa.us>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 51+ messages in thread
From: Tom Lane @ 2026-04-16 17:47 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Peter Eisentraut <peter@eisentraut.org>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Heikki Linnakangas <hlinnaka@iki.fi> writes:
> On 16/04/2026 17:37, Tom Lane wrote:
>> Not excited about making massive changes for this.
> Having all three would be a very localized change in postgres.h.
Sure, but *using* them in a consistent way would be invasive.
>> I remain far less certain than Peter is that this discussion has
>> anything to do with why Coverity is complaining about
>> ginExtractEntries. I still think we should make some minimum-effort
>> change to see if the complaint goes away before expending a lot of
>> brain cells on choosing a final fix.
> I think I'm going to commit my proposal to turn PointerGetDatum() back
> into a macro, and see if that makes Coverity happy. Then we'll know, and
> we can decide on the next steps. Any objections?
WFM.
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-17 19:21 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-17 19:21 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter Eisentraut <peter@eisentraut.org>; David Geier <geidav.pg@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
On 16/04/2026 20:47, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> On 16/04/2026 17:37, Tom Lane wrote:
>>> Not excited about making massive changes for this.
>
>> Having all three would be a very localized change in postgres.h.
>
> Sure, but *using* them in a consistent way would be invasive.
>
>>> I remain far less certain than Peter is that this discussion has
>>> anything to do with why Coverity is complaining about
>>> ginExtractEntries. I still think we should make some minimum-effort
>>> change to see if the complaint goes away before expending a lot of
>>> brain cells on choosing a final fix.
>
>> I think I'm going to commit my proposal to turn PointerGetDatum() back
>> into a macro, and see if that makes Coverity happy. Then we'll know, and
>> we can decide on the next steps. Any objections?
>
> WFM.
Grepping for PointerGetDatum(), there are a bunch of wrappers of it for
specific types, like:
static inline Datum CStringGetDatum(const char *X)
static inline Datum NumericGetDatum(Numeric X)
Most are marked "const". These all potentially have the same problem,
but I think for these it is a good assumption that resulting Datum will
not be used to modify *X, so we can leave them alone. I guess we didn't
do that for NumericGetDatum just because the Numeric typedef doesn't
allow that.
There's also:
static inline Datum fetch_att(const void *T, bool attbyval, int attlen)
The "const" seems reasonable on that too.
This is an interesting case:
static inline Datum
EOHPGetRWDatum(const struct ExpandedObjectHeader *eohptr)
{
return PointerGetDatum(eohptr->eoh_rw_ptr);
}
That RW stands for read/write, which sounds alarming. But the returned
datum points to eohptr->eoh_rw_ptr rather than *eohptr itself, so I
think the 'const' is correct here after all.
So, pushed a commit that changes just PointerGetDatum() itself, leaving
all those others alone.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-22 21:11 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 0 replies; 51+ messages in thread
From: Heikki Linnakangas @ 2026-04-22 21:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers; David Geier <geidav.pg@gmail.com>
On 17/04/2026 22:21, Heikki Linnakangas wrote:
> On 16/04/2026 20:47, Tom Lane wrote:
>> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>>> On 16/04/2026 17:37, Tom Lane wrote:
>>>> Not excited about making massive changes for this.
>>
>>> Having all three would be a very localized change in postgres.h.
>>
>> Sure, but *using* them in a consistent way would be invasive.
>>
>>>> I remain far less certain than Peter is that this discussion has
>>>> anything to do with why Coverity is complaining about
>>>> ginExtractEntries. I still think we should make some minimum-effort
>>>> change to see if the complaint goes away before expending a lot of
>>>> brain cells on choosing a final fix.
>>
>>> I think I'm going to commit my proposal to turn PointerGetDatum() back
>>> into a macro, and see if that makes Coverity happy. Then we'll know, and
>>> we can decide on the next steps. Any objections?
>>
>> WFM.
>
> ...
>
> So, pushed a commit that changes just PointerGetDatum() itself, leaving
> all those others alone.
As we thought, this made the Coverity warning go away.
I'm happy with the status quo in master, but if we want to introduce new
ConstPointerGetDatum() or NonConstPointerGetDatum() variants instead of
the macro, now is the time to do it.
For backbranches, IMHO we should go with the macro. It's a little scary
to replace such a widely used function as PointerGetDatum() in
back-branches, but I do think this should be fixed. Introducing new
variants doesn't seems even less backpatchable.
- Heikki
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-04-22 21:26 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-04-22 21:26 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: pgsql-hackers
On 14.04.2026 16:24, David Geier wrote:
> Attached are the remaining patches (previously 0003 and 0005) rebased on
> latest master. Currently, there's no radix sort variant for the unsigned
> char case. Do we care about this case or is it fine if that case runs
> slower?
>
> The following perf profiles show that trigram_qsort() goes from ~34%
> down to ~7% with the radix sort optimization. The optimized run also
> includes the btint4cmp() optimization. Without that the result would be
> even better.
>
> With that change we could move on and tackle optimizing
>
> 1. 41.52% generate_trgm_only() by e.g. using an ASCII fast-patch
> 2. 32.72% ginInsertBAEntries() by no longer using the RB-tree but
> e.g. also the radix sort
Attached is the rebased patch set as well as a new patch that optimizes
ginInsertBAEntries(). Performance improvements are as follows, measured
with the same benchmark I used in the first mail of this thread.
Runtimes and deltas are in milliseconds.
Code | movies | delta | lineitem | delta
-----------------------------------|--------|--------|------------------
master | 11,160 | - | 248,146 | -
v7-0001-Make-btint4cmp-branchless | 9,509 | 1,651 | 236,760 | 11,386
v7-0002-Use-radix-sort | 6,123 | 3,386 | 214,632 | 22,128
v7-0003-Replace-RB-tree | 4,755 | 1,368 | 144,252 | 70,380
I optimized ginInsertBAEntries() by replacing the red-black tree by a
hash map (simplehash.h) on the keys (e.g. trigrams), where each hash map
entry contains an item pointer list with the row TIDs the key occurs in.
This is in the spirit of the original red-black tree implementation but
this way the deduplication of each key is handled in O(1), instead of
O(log(num_unique_keys)). This reduces the overall runtime complexity from
O(num_total_keys * log(num_unique_keys))
->
O(num_total_keys + num_unique_keys * log(num_unique_keys))
As there are normally a lot less keys than rows, this is complexity-wise
preferable. And even if all keys are unique, the rebalancing operations
are pretty expensive and in reality outweigh the slightly better
complexity in the worst-case.
- The sorting of the item pointer lists for each key stays as is, except
for that I switched to sort_template.h.
- The red-black tree code in rbtree.c/.h is now completely unused and
could be removed. Thoughts?
- The size on disk changed very slightly. I think this is because the
memory accounting is slightly different and with that slightly more or
less keys / item pointers can be processed in one go.
- FWICS, we can use datumIsEqual() and datum_image_hash() in the hash
map because as of today, the GIN index code doesn't work properly with
non-deterministic collations / non-image-equal types, e.g. see [1].
- I also simplified ginInsertBAEntries(). Previously, it used an
insertion order that minimizes the number of RB-tree rebalancing
operations for sorted inputs. With the hash map approach, it's better
to insert in sort order as this increases the likelihood of hitting
the same hash map entry again for equal keys.
- This patch set also improves parallel GIN index builds as the parallel
code builds on top of ginInsertBAEntries().
- Regression tests pass and gin_index_check() from amcheck couldn't find
an error in the data structures.
To be fair: 0001 is less interesting after removing the RB-tree because
a lot less key comparisons are being performed. I still think it makes
sense to merge it as it should also accelerate e.g. B-tree traversals.
With these changes the number one bottleneck becomes the trigram
generation in generate_trgm_only(). The following perf profile nicely
shows that:
- 99.89% 0.00% postgres postgres [.] ginbuild
ginbuild
table_index_build_scan (inlined)
- heapam_index_build_range_scan
- 94.05% ginBuildCallback
- 86.48% ginHeapTupleBulkInsert
- 68.57% ginExtractEntries
- 64.69% FunctionCall3Coll
- gin_extract_value_trgm
- 62.75% generate_trgm
- 39.57% generate_trgm_only
+ 18.53% str_tolower
+ 16.91% find_word (inlined)
+ 1.36% make_trigrams (inlined)
0.66% __strlen_avx2
+ 18.49% trigram_qsort (inlined)
+ 4.17% trigram_qunique (inlined)
0.79% trgm2int
+ 3.33% qsort_arg_entries
- 17.28% ginInsertBAEntries
- ginInsertBAEntry (inlined)
- 8.04% ginbuild_insert (inlined)
- 4.89% ginbuild_insert_hash_internal (inlined)
1.25% gin_equal_key (inlined)
0.68% ginbuild_initial_bucket (inlined)
+ 3.14% gin_hash_key (inlined)
+ 2.45% AllocSetRealloc
+ 0.85% asm_exc_page_fault
0.52% getDatumCopy (inlined)
+ 6.00% ginEntryInsert
+ 1.19% ginBeginBAScan
+ 2.99% heap_getnext
[1]
https://www.postgresql.org/message-id/flat/8ef4899c4acfebca45cc6c042a6dc611d25ffab1.camel%40cybertec...
--
David Geier
Attachments:
[text/x-patch] v7-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch (14.6K, ../../10395cb6-a804-49ac-b2f1-3f98d7204359@gmail.com/2-v7-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch)
download | inline diff:
From c3fb626cae56bcb620af02be55dd18ccab97da72 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v7 3/3] Replace RB-tree with hash map and sort in GIN index
---
src/backend/access/gin/ginbulk.c | 343 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
2 files changed, 172 insertions(+), 195 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..e2d23786225 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData * items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData * items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,109 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
-
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
+
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ if (category == GIN_CAT_NORM_KEY)
+ hashkey.key = getDatumCopy(accum, attnum, key);
+ else
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
- /*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
- */
- if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, sizeof(ItemPointerData) * new_allocated);
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +264,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 6725ee2839f..39c015b1355 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
--
2.51.0
[text/x-patch] v7-0002-Optimize-generate_trgm-with-radix-sort.patch (2.2K, ../../10395cb6-a804-49ac-b2f1-3f98d7204359@gmail.com/3-v7-0002-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From eb8f6c90152ce976161b04c9d540f338a0640b2a Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v7 2/3] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 59 +++++++++++++++++++++++++++++++++------
1 file changed, 51 insertions(+), 8 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 3fd087852e7..f5dc5242c7d 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,13 +226,56 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
@@ -247,7 +290,7 @@ static void
trigram_qsort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ radix_sort_trigrams_signed(array, n);
else
trigram_qsort_unsigned(array, n, sizeof(trgm));
}
--
2.51.0
[text/x-patch] v7-0001-Make-btint4cmp-branchless.patch (1.0K, ../../10395cb6-a804-49ac-b2f1-3f98d7204359@gmail.com/4-v7-0001-Make-btint4cmp-branchless.patch)
download | inline diff:
From abeea158b0f58344abb085f914d5cc5f804395c8 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v7 1/3] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 1d343377e98..ac16e3d993d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -203,12 +204,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-01 11:05 David Geier <geidav.pg@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-09-01 11:05 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: pgsql-hackers
Hi!
I've rebased the patch set on latest master.
I'm hoping we can make some progress with the patch set, given that it
gives a huge performance improvement, allowing to create GIN indexes on
much bigger tables.
@Heikki and @Matthias: anything specific missing from your point-of-view
that is blocking this patch set from moving forward?
> Attached is the rebased patch set as well as a new patch that optimizes
> ginInsertBAEntries(). Performance improvements are as follows, measured
> with the same benchmark I used in the first mail of this thread.
> Runtimes and deltas are in milliseconds.
>
> Code | movies | delta | lineitem | delta
> -----------------------------------|--------|--------|------------------
> master | 11,160 | - | 248,146 | -
> v7-0001-Make-btint4cmp-branchless | 9,509 | 1,651 | 236,760 | 11,386
> v7-0002-Use-radix-sort | 6,123 | 3,386 | 214,632 | 22,128
> v7-0003-Replace-RB-tree | 4,755 | 1,368 | 144,252 | 70,380
For details of the implementation see my previous mail.
--
David Geier
Attachments:
[text/x-patch] v8-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch (14.6K, ../../ec7e4221-881d-4026-af44-484982b9f101@gmail.com/2-v8-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch)
download | inline diff:
From 3823e95e3f160ebb1d538c751841920c01274254 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v8 3/3] Replace RB-tree with hash map and sort in GIN index
---
src/backend/access/gin/ginbulk.c | 343 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
2 files changed, 172 insertions(+), 195 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..e2d23786225 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData * items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData * items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,109 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
-
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
+
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ if (category == GIN_CAT_NORM_KEY)
+ hashkey.key = getDatumCopy(accum, attnum, key);
+ else
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
- /*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
- */
- if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, sizeof(ItemPointerData) * new_allocated);
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +264,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3c5fd6ba817..df6b44796c6 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
--
2.51.0
[text/x-patch] v8-0002-Optimize-generate_trgm-with-radix-sort.patch (2.2K, ../../ec7e4221-881d-4026-af44-484982b9f101@gmail.com/3-v8-0002-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From 0a8360ec3c5414a0cf5e92ec70dfcad610cc912e Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v8 2/3] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 59 +++++++++++++++++++++++++++++++++------
1 file changed, 51 insertions(+), 8 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 22bcc3c3361..1481fa0ceaa 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,13 +226,56 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char FlipSign(char x)
+{
+ return x^0x80;
+}
+
+static void radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
+ freqs[j][FlipSign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
+ memcpy(starts[FlipSign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
@@ -247,7 +290,7 @@ static void
trigram_qsort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ radix_sort_trigrams_signed(array, n);
else
trigram_qsort_unsigned(array, n, sizeof(trgm));
}
--
2.51.0
[text/x-patch] v8-0001-Make-btint4cmp-branchless.patch (1.0K, ../../ec7e4221-881d-4026-af44-484982b9f101@gmail.com/4-v8-0001-Make-btint4cmp-branchless.patch)
download | inline diff:
From affbd8e92b89ceddf3fc1e32a2f6ab45fc538e51 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v8 1/3] Make btint4cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 4e3a3a0f7ce..7f2a08843d3 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-01 15:21 Japin Li <japinli@hotmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Japin Li @ 2026-09-01 15:21 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi, David
Thanks for this—looks like a good improvement.
On Tue, 01 Sep 2026 at 13:05, David Geier <geidav.pg@gmail.com> wrote:
> Hi!
>
> I've rebased the patch set on latest master.
>
> I'm hoping we can make some progress with the patch set, given that it
> gives a huge performance improvement, allowing to create GIN indexes on
> much bigger tables.
>
> @Heikki and @Matthias: anything specific missing from your point-of-view
> that is blocking this patch set from moving forward?
>
>> Attached is the rebased patch set as well as a new patch that optimizes
>> ginInsertBAEntries(). Performance improvements are as follows, measured
>> with the same benchmark I used in the first mail of this thread.
>> Runtimes and deltas are in milliseconds.
>>
>> Code | movies | delta | lineitem | delta
>> -----------------------------------|--------|--------|------------------
>> master | 11,160 | - | 248,146 | -
>> v7-0001-Make-btint4cmp-branchless | 9,509 | 1,651 | 236,760 | 11,386
>> v7-0002-Use-radix-sort | 6,123 | 3,386 | 214,632 | 22,128
>> v7-0003-Replace-RB-tree | 4,755 | 1,368 | 144,252 | 70,380
>
> For details of the implementation see my previous mail.
Here are some comments on v8 patches.
v8-0001
=======
1.
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
While we are in this area, would it make sense to apply the same treatment to
btint8cmp() using pg_cmp_s64()?
v8-0002
=======
1.
+static inline unsigned char FlipSign(char x)
Coding style nit: suggest formatting this as:
+static inline unsigned char
+FlipSign(char x)
2.
+static void radix_sort_trigrams_signed(trgm *trg, int count)
Same as above.
3.
+ for (int i=0; i<count; i++)
+ for (int j=0; j<3; j++)
Spaces are required between operators and their operands.
4.
+ for (int i=2; i>=0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j=0; j<256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j=0; j<count; j++)
Same as above.
v8-0003
=======
1.
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
...
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData * items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData * items;
+ uint32 numItems;
+} GinSortEntry;
Since this patch introduces new typedefs, GinHashKey, GinHashEntry and
GinSortEntry, typedefs.list should probably be updated as well.
2.
+ ItemPointerData * items;
This is inconsistent with our coding style.
3.
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
Remove GinEntryAccumulator from typedefs.list as well.
>
> --
> David Geier
--
Regards,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-02 09:16 David Geier <geidav.pg@gmail.com>
parent: Japin Li <japinli@hotmail.com>
0 siblings, 2 replies; 51+ messages in thread
From: David Geier @ 2026-09-02 09:16 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi!
> Thanks for this—looks like a good improvement.
Thanks for reviewing the patch. Attached is v9 with all reviewing
comments from below addressed.
> Here are some comments on v8 patches.
>
> v8-0001
> =======
>
> 1.
> @@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
> int32 a = PG_GETARG_INT32(0);
> int32 b = PG_GETARG_INT32(1);
>
> - if (a > b)
> - PG_RETURN_INT32(A_GREATER_THAN_B);
> - else if (a == b)
> - PG_RETURN_INT32(0);
> - else
> - PG_RETURN_INT32(A_LESS_THAN_B);
> + PG_RETURN_INT32(pg_cmp_s32(a, b));
> }
>
> While we are in this area, would it make sense to apply the same treatment to
> btint8cmp() using pg_cmp_s64()?
Done. If there's no consensus that this optimization won't cause
regressions we can also split it out from this patchset. I would then
open a new thread with additional testing.
> v8-0002
> =======
>
> 1.
> +static inline unsigned char FlipSign(char x)
>
> Coding style nit: suggest formatting this as:
>
> +static inline unsigned char
> +FlipSign(char x)
>
> 2.
> +static void radix_sort_trigrams_signed(trgm *trg, int count)
>
> Same as above.
>
> 3.
> + for (int i=0; i<count; i++)
> + for (int j=0; j<3; j++)
>
> Spaces are required between operators and their operands.
>
> 4.
> + for (int i=2; i>=0; i--)
> + {
> + trgm *old_from = from;
> + trgm *next = to;
> +
> + for (int j=0; j<256; j++)
> + {
> + starts[j] = next;
> + next += freqs[i][j];
> + }
> +
> + for (int j=0; j<count; j++)
>
> Same as above.
Done. Also renamed FlipSign() to flip_sign() for consistency.
> v8-0003
> =======
>
> 1.
> +typedef struct GinHashKey
> {
> - GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
> - const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
> - BuildAccumulator *accum = (BuildAccumulator *) arg;
> + OffsetNumber attnum;
> + GinNullCategory category;
> + Datum key;
> +} GinHashKey;
> ...
> +typedef struct GinHashEntry
> +{
> + GinHashKey hashkey;
> + uint32 hash;
> + char status;
> + ItemPointerData * items;
> + uint32 numItems;
> + uint32 allocatedItems;
> +} GinHashEntry;
> +
> +typedef struct GinSortEntry
> +{
> + GinHashKey hashkey;
> + ItemPointerData * items;
> + uint32 numItems;
> +} GinSortEntry;
>
> Since this patch introduces new typedefs, GinHashKey, GinHashEntry and
> GinSortEntry, typedefs.list should probably be updated as well.
Done.
> 2.
> + ItemPointerData * items;
>
> This is inconsistent with our coding style.
>
> 3.
> -typedef struct GinEntryAccumulator
> -{
> - RBTNode rbtnode;
> - Datum key;
> - GinNullCategory category;
> - OffsetNumber attnum;
> - bool shouldSort;
> - ItemPointerData *list;
> - uint32 maxcount; /* allocated size of list[] */
> - uint32 count; /* current number of list[] entries */
> -} GinEntryAccumulator;
>
> Remove GinEntryAccumulator from typedefs.list as well.
Done.
I realized that one elog(ERROR) got removed and another one with a
different message got added. I haven't updated the translation files
because, judging from the git log messages, that is done separately.
How to best go about complying to the existing code style? I've been
under the impression that especially indentation is mostly fixed up
retroactively by pgindent. Do you run pgindent on the patch set prior to
submitting the patch?
--
David Geier
Attachments:
[text/x-patch] v9-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch (15.1K, ../../bef2dece-b600-452d-b375-afca1c56c5ed@gmail.com/2-v9-0003-Replace-RB-tree-with-hash-map-and-sort-in-GIN-ind.patch)
download | inline diff:
From 8b216c8ab4823bc811aec8e791235befa0719565 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v9 3/3] Replace RB-tree with hash map and sort in GIN index
---
src/backend/access/gin/ginbulk.c | 343 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
src/tools/pgindent/typedefs.list | 4 +-
3 files changed, 175 insertions(+), 196 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..aa3e66da797 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData *items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData *items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,109 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
-
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
+
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ if (category == GIN_CAT_NORM_KEY)
+ hashkey.key = getDatumCopy(accum, attnum, key);
+ else
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
- /*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
- */
- if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, mul_size(sizeof(ItemPointerData), new_allocated));
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +264,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3c5fd6ba817..df6b44796c6 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c546b3d6375..fe82651e86d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1121,7 +1121,8 @@ GinBuildShared
GinBuildState
GinChkVal
GinEntries
-GinEntryAccumulator
+GinHashEntry
+GinHashKey
GinIndexStat
GinLeader
GinMetaPageData
@@ -1141,6 +1142,7 @@ GinScanKeyData
GinScanOpaque
GinScanOpaqueData
GinSegmentInfo
+GinSortEntry
GinState
GinStatsData
GinTernaryValue
--
2.51.0
[text/x-patch] v9-0002-Optimize-generate_trgm-with-radix-sort.patch (2.2K, ../../bef2dece-b600-452d-b375-afca1c56c5ed@gmail.com/3-v9-0002-Optimize-generate_trgm-with-radix-sort.patch)
download | inline diff:
From acb57bd236d6c4e5188a71824a8c162af423a08c Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v9 2/3] Optimize generate_trgm() with radix sort
---
contrib/pg_trgm/trgm_op.c | 61 ++++++++++++++++++++++++++++++++++-----
1 file changed, 53 insertions(+), 8 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 22bcc3c3361..cdb8d7d871b 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,13 +226,58 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char
+flip_sign(char x)
+{
+ return x ^ 0x80;
+}
+
+static void
+radix_sort_trigrams_signed(trgm *trg, int count)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ int freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (int i = 0; i < count; i++)
+ for (int j = 0; j < 3; j++)
+ freqs[j][flip_sign(trg[i][j])]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the is "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i = 2; i >= 0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (int j = 0; j < 256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (int j = 0; j < count; j++)
+ memcpy(starts[flip_sign(from[j][i])]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
#define ST_SORT trigram_qsort_unsigned
#define ST_ELEMENT_TYPE_VOID
@@ -247,7 +292,7 @@ static void
trigram_qsort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ radix_sort_trigrams_signed(array, n);
else
trigram_qsort_unsigned(array, n, sizeof(trgm));
}
--
2.51.0
[text/x-patch] v9-0001-Make-btint4cmp-and-btint8cmp-branchless.patch (1.3K, ../../bef2dece-b600-452d-b375-afca1c56c5ed@gmail.com/4-v9-0001-Make-btint4cmp-and-btint8cmp-branchless.patch)
download | inline diff:
From 3fdb416695504f7d31cd1913f545145d2ed2d2f1 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v9 1/3] Make btint4cmp() and btint8cmp() branchless
---
src/backend/access/nbtree/nbtcompare.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 4e3a3a0f7ce..80dec200a3d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
@@ -262,12 +258,7 @@ btint8cmp(PG_FUNCTION_ARGS)
int64 a = PG_GETARG_INT64(0);
int64 b = PG_GETARG_INT64(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s64(a, b));
}
Datum
--
2.51.0
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-02 15:15 Japin Li <japinli@hotmail.com>
parent: David Geier <geidav.pg@gmail.com>
1 sibling, 1 reply; 51+ messages in thread
From: Japin Li @ 2026-09-02 15:15 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi!
On Wed, 02 Sep 2026 at 11:16, David Geier <geidav.pg@gmail.com> wrote:
> Hi!
>
>> Thanks for this—looks like a good improvement.
>
> Thanks for reviewing the patch. Attached is v9 with all reviewing
> comments from below addressed.
>
Thanks for updating the patches.
>> Here are some comments on v8 patches.
>>
>> v8-0001
>> =======
>>
>> 1.
>> @@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
>> int32 a = PG_GETARG_INT32(0);
>> int32 b = PG_GETARG_INT32(1);
>>
>> - if (a > b)
>> - PG_RETURN_INT32(A_GREATER_THAN_B);
>> - else if (a == b)
>> - PG_RETURN_INT32(0);
>> - else
>> - PG_RETURN_INT32(A_LESS_THAN_B);
>> + PG_RETURN_INT32(pg_cmp_s32(a, b));
>> }
>>
>> While we are in this area, would it make sense to apply the same treatment to
>> btint8cmp() using pg_cmp_s64()?
>
> Done. If there's no consensus that this optimization won't cause
> regressions we can also split it out from this patchset. I would then
> open a new thread with additional testing.
>
>> v8-0002
>> =======
>>
>> 1.
>> +static inline unsigned char FlipSign(char x)
>>
>> Coding style nit: suggest formatting this as:
>>
>> +static inline unsigned char
>> +FlipSign(char x)
>>
>> 2.
>> +static void radix_sort_trigrams_signed(trgm *trg, int count)
>>
>> Same as above.
>>
>> 3.
>> + for (int i=0; i<count; i++)
>> + for (int j=0; j<3; j++)
>>
>> Spaces are required between operators and their operands.
>>
>> 4.
>> + for (int i=2; i>=0; i--)
>> + {
>> + trgm *old_from = from;
>> + trgm *next = to;
>> +
>> + for (int j=0; j<256; j++)
>> + {
>> + starts[j] = next;
>> + next += freqs[i][j];
>> + }
>> +
>> + for (int j=0; j<count; j++)
>>
>> Same as above.
>
> Done. Also renamed FlipSign() to flip_sign() for consistency.
>
>> v8-0003
>> =======
>>
>> 1.
>> +typedef struct GinHashKey
>> {
>> - GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
>> - const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
>> - BuildAccumulator *accum = (BuildAccumulator *) arg;
>> + OffsetNumber attnum;
>> + GinNullCategory category;
>> + Datum key;
>> +} GinHashKey;
>> ...
>> +typedef struct GinHashEntry
>> +{
>> + GinHashKey hashkey;
>> + uint32 hash;
>> + char status;
>> + ItemPointerData * items;
>> + uint32 numItems;
>> + uint32 allocatedItems;
>> +} GinHashEntry;
>> +
>> +typedef struct GinSortEntry
>> +{
>> + GinHashKey hashkey;
>> + ItemPointerData * items;
>> + uint32 numItems;
>> +} GinSortEntry;
>>
>> Since this patch introduces new typedefs, GinHashKey, GinHashEntry and
>> GinSortEntry, typedefs.list should probably be updated as well.
>
> Done.
>
>> 2.
>> + ItemPointerData * items;
>>
>> This is inconsistent with our coding style.
>>
>> 3.
>> -typedef struct GinEntryAccumulator
>> -{
>> - RBTNode rbtnode;
>> - Datum key;
>> - GinNullCategory category;
>> - OffsetNumber attnum;
>> - bool shouldSort;
>> - ItemPointerData *list;
>> - uint32 maxcount; /* allocated size of list[] */
>> - uint32 count; /* current number of list[] entries */
>> -} GinEntryAccumulator;
>>
>> Remove GinEntryAccumulator from typedefs.list as well.
>
> Done.
>
> I realized that one elog(ERROR) got removed and another one with a
> different message got added. I haven't updated the translation files
> because, judging from the git log messages, that is done separately.
>
> How to best go about complying to the existing code style? I've been
> under the impression that especially indentation is mostly fixed up
> retroactively by pgindent. Do you run pgindent on the patch set prior to
> submitting the patch?
I'm relying on the IDE's auto-formatting, which fixes most coding style issues.
The committer will run pgindent anyway [1] — so I didn't run it beforehand.
[1] https://wiki.postgresql.org/wiki/Committing_checklist
--
Regards,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-03 07:41 David Geier <geidav.pg@gmail.com>
parent: Japin Li <japinli@hotmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: David Geier @ 2026-09-03 07:41 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers
Hi!
>> Thanks for reviewing the patch. Attached is v9 with all reviewing
>> comments from below addressed.
>>
>
> Thanks for updating the patches.
Can you add yourself as reviewer to the commit fest entry? See [1].
> I'm relying on the IDE's auto-formatting, which fixes most coding style issues.
>
> The committer will run pgindent anyway [1] — so I didn't run it beforehand.
That's what I thought. Thanks for the pointer to the committing checklist.
--
David Geier
[1] https://commitfest.postgresql.org/patch/6418/.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-08 14:58 Matthias van de Meent <boekewurm+postgres@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
1 sibling, 1 reply; 51+ messages in thread
From: Matthias van de Meent @ 2026-09-08 14:58 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
On Wed, 2 Sept 2026 at 11:16, David Geier <geidav.pg@gmail.com> wrote:
>
> Hi!
>
> > Thanks for this—looks like a good improvement.
>
> Thanks for reviewing the patch. Attached is v9 with all reviewing
> comments from below addressed.
Please add commit descriptions; the current patches don't have much
other than their subjects which don't describe any rationale for the
changes applied.
0001: LGTM
0002:
1. radix_sort_trigrams_signed has an `int count` argument, but the
caller trigram_qsort uses size_t.
2. This increases the memory requirements of trigram_qsort by a huge margin.
Could you change the radixsort to operate in-place, so that the new
buffer is not needed?
3. The implementation for trigram_qsort_unsigned has not been adjusted
nor replaced, and so keeps the same old performance that
trigram_qsort_signed had.
Please make sure to also adjust that implementation.
0003:
1. ginInsertBAEntry leaks the copied key Datum if the type is by-ref
and already present in the accumulator.
I think specifying HASH_KEYCOPY to only copying the Datum key if an
entry is already present is more appropriate.
2. The comment for ginInsertBAEntries was completely removed, rather
than updated to the new workings.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-09 08:49 David Geier <geidav.pg@gmail.com>
parent: Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: David Geier @ 2026-09-09 08:49 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
Hi Matthias!
Thanks a lot for your review.
> Please add commit descriptions; the current patches don't have much
> other than their subjects which don't describe any rationale for the
> changes applied.
Done.
> 0001: LGTM
> 0002:
> 1. radix_sort_trigrams_signed has an `int count` argument, but the
> caller trigram_qsort uses size_t.
Changed count argument to size_t.
> 2. This increases the memory requirements of trigram_qsort by a huge margin.
> Could you change the radixsort to operate in-place, so that the new
> buffer is not needed?
Trigram radix sort is only called from generate_trgm() and
generate_wildcard_trgm() which are both used to extract the unique
trigrams from a _single_ string value.
We don't radix sort trigrams of multiple string values. Hence, the
maximum increase in memory, while building the GIN index, is in the size
of the longest string encountered, not in the number of rows.
Given that in-place radix sort is a lot more complex and slower, I think
the current tradeoff is fine.
Beyond that, other GIN index functions also allocate extra memory on a
per-value basis which shows that this should not be a problem in
practice (e.g. ginarrayextract(), gin_extract_value_trgm(), ...).
> 3. The implementation for trigram_qsort_unsigned has not been adjusted
> nor replaced, and so keeps the same old performance that
> trigram_qsort_signed had.
> Please make sure to also adjust that implementation.
Done.
I laid out the code such that the compiler has the possibility to fully
inline both variants to get rid of the extra code in radix_key() for
flipping the sign bit in the unsigned case. But even if it doesn't,
radix_key() can be branchless and the performance is anyway dominated by
memory traffic.
> 0003:
>
> 1. ginInsertBAEntry leaks the copied key Datum if the type is by-ref
> and already present in the accumulator.
Good catch!
They weren't leaked for good but would have gotten cleaned up with the
next batch. But they would have unnecessarily increased the memory
footprint. Especially, as the number of unique trigrams is typically low
and hence we often hit the "already present in the accumulator" path.
> I think specifying HASH_KEYCOPY to only copying the Datum key if an
> entry is already present is more appropriate.
I cannot use HASH_KEYCOPY because I'm using simplehash.h. I changed it
to use the original key for doing the lookup and only create the copy in
the !found path. This is also how the old code did it. Changing the key
of an already inserted hashmap entry is save because we overwrite it
with a copy. So the hash function will keep returning the same position
for it.
> 2. The comment for ginInsertBAEntries was completely removed, rather
> than updated to the new workings.
Yes, because the comment no longer applies. All of it was specific to
using a red-black tree. ginInsertBAEntries() is now merely syntactic
sugar. I thought about removing the function completely but it's used in
a couple of places and without it, the call sites would get slightly
more ugly.
Would you like to see anything specific in the comment?
Attached is the updated patch, rebased on latest master. With 0003,
rbtree.h/.c are completely unused and could be removed as well,
including the test code.
--
David Geier
From cf9dd48306cf0399ea45d540b55aacf744210cef Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v10 1/3] Use branchless comparisons in btint4cmp and btint8cmp
Use the common pg_cmp_s32() and pg_cmp_s64() helpers to implement the
built-in B-tree comparison functions for int4 and int8.
The previous implementations used conditional branches to distinguish
less-than, equal, and greater-than values. The common comparison helpers
perform the same three-way comparison without data-dependent branches,
which can improve performance for workloads involving frequent integer
comparisons while preserving the required comparator result semantics.
btint4cmp() and btint8cmp() are PostgreSQL-callable functions invoked
through the function manager. They are not inlined at their call sites,
so replacing the original conditional implementation does not prevent a
compiler from optimizing an inline comparison in contexts where it can
see and better optimize the surrounding code. In other words, this
change affects the function-manager call path without imposing a
performance regression on callers for which the comparison could
otherwise have been inlined.
The comparison helpers also provide the appropriate handling for the
full ranges of int32 and int64 values without relying on subtraction,
which could overflow for values near the type limits.
---
src/backend/access/nbtree/nbtcompare.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 4e3a3a0f7ce..80dec200a3d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
@@ -262,12 +258,7 @@ btint8cmp(PG_FUNCTION_ARGS)
int64 a = PG_GETARG_INT64(0);
int64 b = PG_GETARG_INT64(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s64(a, b));
}
Datum
--
2.50.1 (Apple Git-155)
From 24f53244f6e8f44957e7cb1ff0e6cdd920aaab19 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v10 2/3] Use radix sort to extract trigrams
Replace the comparison-based sort used by generate_trgm() and
generate_wildcard_trgm() with a three-pass radix sort.
Trigrams consist of three bytes, so their keys have a fixed and very
small width. A radix sort can therefore order them in linear time with
respect to the number of trigrams, avoiding the repeated comparator calls
and recursive partitioning performed by qsort.
The implementation preserves the existing behavior on platforms where
char is signed by flipping the most significant bit before sorting.
The radix sort requires a temporary buffer, increasing the memory
footprint while trigrams are being extracted from an input string.
However, the sort is performed separately for each string and is never
applied across multiple strings at once. Consequently, the additional
memory is limited to the processing of the current string and should
have a negligible effect on the overall memory footprint of a GIN index
build.
The resulting order remains compatible with trigram deduplication and
does not change the generated trigram sets.
---
contrib/pg_trgm/trgm_op.c | 78 ++++++++++++++++++++++++++++-----------
1 file changed, 57 insertions(+), 21 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 22bcc3c3361..bfda0df9fd4 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,33 +226,69 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
-#define ST_SORT trigram_qsort_unsigned
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char
+radix_key(char x, bool char_is_signed)
+{
+ return char_is_signed ? x ^ 0x80 : x;
+}
+
+static inline void
+trigram_radix_sort_with_signedness(trgm *trg, size_t count, bool char_is_signed)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ size_t freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (size_t i = 0; i < count; i++)
+ for (size_t j = 0; j < 3; j++)
+ freqs[j][radix_key(trg[i][j], char_is_signed)]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i = 2; i >= 0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (size_t j = 0; j < 256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (size_t j = 0; j < count; j++)
+ memcpy(starts[radix_key(from[j][i], char_is_signed)]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
/* Sort an array of trigrams, handling signedness correctly */
static void
-trigram_qsort(trgm *array, size_t n)
+trigram_radix_sort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ trigram_radix_sort_with_signedness(array, n, true);
else
- trigram_qsort_unsigned(array, n, sizeof(trgm));
+ trigram_radix_sort_with_signedness(array, n, false);
}
-
/*
* Compare two trigrams for equality. This has the same signature as
* comparison functions used for sorting, so that this can be used with
@@ -612,7 +648,7 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- trigram_qsort(GETARR(trg), len);
+ trigram_radix_sort(GETARR(trg), len);
len = trigram_qunique(GETARR(trg), len);
}
@@ -1143,7 +1179,7 @@ generate_wildcard_trgm(const char *str, int slen)
len = arr.length;
if (len > 1)
{
- trigram_qsort(GETARR(trg), len);
+ trigram_radix_sort(GETARR(trg), len);
len = trigram_qunique(GETARR(trg), len);
}
--
2.50.1 (Apple Git-155)
From d24a1ff663fcaf0d8adf3661dbc65efa3dddf403 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v10 3/3] Replace GIN build accumulator RB-tree with a hashmap
Replace the red-black tree used by ginInsertBAEntries() with a hash table
based on simplehash.h. This keeps the same basic accumulation strategy
as the previous implementation while changing key lookup and deduplication
from O(log(num_unique_keys)) tree operations to expected O(1) hash-table
operations. As a result, the overall complexity changes from
O(num_total_keys * log(num_unique_keys))
to
O(num_total_keys + num_unique_keys * log(num_unique_keys))
The latter is preferable for the usual case, where the number of unique
keys is much smaller than the number of rows. Even when most or all keys
are unique, the RB-tree rebalancing operations are sufficiently
expensive that the theoretical worst-case advantage of the tree does not
necessarily translate into better runtime.
The item pointer lists associated with each key continue to be sorted
before the accumulated entries are emitted. Use sort_template.h for this
sorting instead of qsort(). The distinct hash entries are copied into an
array and sorted using the existing GIN key comparison function so that
the output order remains unchanged.
ginInsertBAEntries() got much simpler as well. The previous implementation
inserted entries in an order intended to minimize red-black tree rebalancing
for sorted inputs. With a hash table, inserting the entries in their original
order is preferable because repeated keys are more likely to be found in the
same hash-table entry.
Use datumIsEqual() and datum_image_hash() for normal GIN keys. This is
consistent with the current GIN implementation, which does not correctly
support non-deterministic collations or types for which logically equal
values are not image-equal.
Because parallel GIN index builds also use ginInsertBAEntries(), this
change improves the accumulation phase of parallel index builds as well.
---
src/backend/access/gin/ginbulk.c | 339 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
src/tools/pgindent/typedefs.list | 4 +-
3 files changed, 175 insertions(+), 192 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..77c12ba932b 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData *items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData *items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,113 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
/*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
+ * Finish initializing new hashmap entry including making a permanent
+ * copy of the key.
*/
if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->hashkey.key = getDatumCopy(accum, attnum, key);
+
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, mul_size(sizeof(ItemPointerData), new_allocated));
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +268,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3c5fd6ba817..df6b44796c6 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1040a65bc14..51f8b52b3a8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1107,7 +1107,8 @@ GinBuildShared
GinBuildState
GinChkVal
GinEntries
-GinEntryAccumulator
+GinHashEntry
+GinHashKey
GinIndexStat
GinLeader
GinMetaPageData
@@ -1127,6 +1128,7 @@ GinScanKeyData
GinScanOpaque
GinScanOpaqueData
GinSegmentInfo
+GinSortEntry
GinState
GinStatsData
GinTernaryValue
--
2.50.1 (Apple Git-155)
Attachments:
[text/plain] v10-0001-Use-branchless-comparisons-in-btint4cmp-and-btin.patch (2.5K, ../../1c90691c-db53-420a-be6c-e0323e6ae9f1@gmail.com/2-v10-0001-Use-branchless-comparisons-in-btint4cmp-and-btin.patch)
download | inline diff:
From cf9dd48306cf0399ea45d540b55aacf744210cef Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v10 1/3] Use branchless comparisons in btint4cmp and btint8cmp
Use the common pg_cmp_s32() and pg_cmp_s64() helpers to implement the
built-in B-tree comparison functions for int4 and int8.
The previous implementations used conditional branches to distinguish
less-than, equal, and greater-than values. The common comparison helpers
perform the same three-way comparison without data-dependent branches,
which can improve performance for workloads involving frequent integer
comparisons while preserving the required comparator result semantics.
btint4cmp() and btint8cmp() are PostgreSQL-callable functions invoked
through the function manager. They are not inlined at their call sites,
so replacing the original conditional implementation does not prevent a
compiler from optimizing an inline comparison in contexts where it can
see and better optimize the surrounding code. In other words, this
change affects the function-manager call path without imposing a
performance regression on callers for which the comparison could
otherwise have been inlined.
The comparison helpers also provide the appropriate handling for the
full ranges of int32 and int64 values without relying on subtraction,
which could overflow for values near the type limits.
---
src/backend/access/nbtree/nbtcompare.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 4e3a3a0f7ce..80dec200a3d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
@@ -262,12 +258,7 @@ btint8cmp(PG_FUNCTION_ARGS)
int64 a = PG_GETARG_INT64(0);
int64 b = PG_GETARG_INT64(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s64(a, b));
}
Datum
--
2.50.1 (Apple Git-155)
[text/plain] v10-0002-Use-radix-sort-to-extract-trigrams.patch (4.3K, ../../1c90691c-db53-420a-be6c-e0323e6ae9f1@gmail.com/3-v10-0002-Use-radix-sort-to-extract-trigrams.patch)
download | inline diff:
From 24f53244f6e8f44957e7cb1ff0e6cdd920aaab19 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v10 2/3] Use radix sort to extract trigrams
Replace the comparison-based sort used by generate_trgm() and
generate_wildcard_trgm() with a three-pass radix sort.
Trigrams consist of three bytes, so their keys have a fixed and very
small width. A radix sort can therefore order them in linear time with
respect to the number of trigrams, avoiding the repeated comparator calls
and recursive partitioning performed by qsort.
The implementation preserves the existing behavior on platforms where
char is signed by flipping the most significant bit before sorting.
The radix sort requires a temporary buffer, increasing the memory
footprint while trigrams are being extracted from an input string.
However, the sort is performed separately for each string and is never
applied across multiple strings at once. Consequently, the additional
memory is limited to the processing of the current string and should
have a negligible effect on the overall memory footprint of a GIN index
build.
The resulting order remains compatible with trigram deduplication and
does not change the generated trigram sets.
---
contrib/pg_trgm/trgm_op.c | 78 ++++++++++++++++++++++++++++-----------
1 file changed, 57 insertions(+), 21 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 22bcc3c3361..bfda0df9fd4 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,33 +226,69 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
-#define ST_SORT trigram_qsort_unsigned
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char
+radix_key(char x, bool char_is_signed)
+{
+ return char_is_signed ? x ^ 0x80 : x;
+}
+
+static inline void
+trigram_radix_sort_with_signedness(trgm *trg, size_t count, bool char_is_signed)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ size_t freqs[3][256];
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (size_t i = 0; i < count; i++)
+ for (size_t j = 0; j < 3; j++)
+ freqs[j][radix_key(trg[i][j], char_is_signed)]++;
+
+ /*
+ * Do the sorting. Start with last character because that's the "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i = 2; i >= 0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ for (size_t j = 0; j < 256; j++)
+ {
+ starts[j] = next;
+ next += freqs[i][j];
+ }
+
+ for (size_t j = 0; j < count; j++)
+ memcpy(starts[radix_key(from[j][i], char_is_signed)]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+}
/* Sort an array of trigrams, handling signedness correctly */
static void
-trigram_qsort(trgm *array, size_t n)
+trigram_radix_sort(trgm *array, size_t n)
{
if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
+ trigram_radix_sort_with_signedness(array, n, true);
else
- trigram_qsort_unsigned(array, n, sizeof(trgm));
+ trigram_radix_sort_with_signedness(array, n, false);
}
-
/*
* Compare two trigrams for equality. This has the same signature as
* comparison functions used for sorting, so that this can be used with
@@ -612,7 +648,7 @@ generate_trgm(char *str, int slen)
*/
if (len > 1)
{
- trigram_qsort(GETARR(trg), len);
+ trigram_radix_sort(GETARR(trg), len);
len = trigram_qunique(GETARR(trg), len);
}
@@ -1143,7 +1179,7 @@ generate_wildcard_trgm(const char *str, int slen)
len = arr.length;
if (len > 1)
{
- trigram_qsort(GETARR(trg), len);
+ trigram_radix_sort(GETARR(trg), len);
len = trigram_qunique(GETARR(trg), len);
}
--
2.50.1 (Apple Git-155)
[text/plain] v10-0003-Replace-GIN-build-accumulator-RB-tree-with-a-has.patch (17.0K, ../../1c90691c-db53-420a-be6c-e0323e6ae9f1@gmail.com/4-v10-0003-Replace-GIN-build-accumulator-RB-tree-with-a-has.patch)
download | inline diff:
From d24a1ff663fcaf0d8adf3661dbc65efa3dddf403 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v10 3/3] Replace GIN build accumulator RB-tree with a hashmap
Replace the red-black tree used by ginInsertBAEntries() with a hash table
based on simplehash.h. This keeps the same basic accumulation strategy
as the previous implementation while changing key lookup and deduplication
from O(log(num_unique_keys)) tree operations to expected O(1) hash-table
operations. As a result, the overall complexity changes from
O(num_total_keys * log(num_unique_keys))
to
O(num_total_keys + num_unique_keys * log(num_unique_keys))
The latter is preferable for the usual case, where the number of unique
keys is much smaller than the number of rows. Even when most or all keys
are unique, the RB-tree rebalancing operations are sufficiently
expensive that the theoretical worst-case advantage of the tree does not
necessarily translate into better runtime.
The item pointer lists associated with each key continue to be sorted
before the accumulated entries are emitted. Use sort_template.h for this
sorting instead of qsort(). The distinct hash entries are copied into an
array and sorted using the existing GIN key comparison function so that
the output order remains unchanged.
ginInsertBAEntries() got much simpler as well. The previous implementation
inserted entries in an order intended to minimize red-black tree rebalancing
for sorted inputs. With a hash table, inserting the entries in their original
order is preferable because repeated keys are more likely to be found in the
same hash-table entry.
Use datumIsEqual() and datum_image_hash() for normal GIN keys. This is
consistent with the current GIN implementation, which does not correctly
support non-deterministic collations or types for which logically equal
values are not image-equal.
Because parallel GIN index builds also use ginInsertBAEntries(), this
change improves the accumulation phase of parallel index builds as well.
---
src/backend/access/gin/ginbulk.c | 339 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
src/tools/pgindent/typedefs.list | 4 +-
3 files changed, 175 insertions(+), 192 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..77c12ba932b 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData *items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData *items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,113 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
/*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
+ * Finish initializing new hashmap entry including making a permanent
+ * copy of the key.
*/
if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->hashkey.key = getDatumCopy(accum, attnum, key);
+
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, mul_size(sizeof(ItemPointerData), new_allocated));
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +268,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3c5fd6ba817..df6b44796c6 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1040a65bc14..51f8b52b3a8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1107,7 +1107,8 @@ GinBuildShared
GinBuildState
GinChkVal
GinEntries
-GinEntryAccumulator
+GinHashEntry
+GinHashKey
GinIndexStat
GinLeader
GinMetaPageData
@@ -1127,6 +1128,7 @@ GinScanKeyData
GinScanOpaque
GinScanOpaqueData
GinSegmentInfo
+GinSortEntry
GinState
GinStatsData
GinTernaryValue
--
2.50.1 (Apple Git-155)
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-09 14:30 Matthias van de Meent <boekewurm+postgres@gmail.com>
parent: David Geier <geidav.pg@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Matthias van de Meent @ 2026-09-09 14:30 UTC (permalink / raw)
To: David Geier <geidav.pg@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
On Wed, 9 Sept 2026 at 10:49, David Geier <geidav.pg@gmail.com> wrote:
>
> Hi Matthias!
>
> Thanks a lot for your review.
>
> > Please add commit descriptions; the current patches don't have much
> > other than their subjects which don't describe any rationale for the
> > changes applied.
>
> Done.
0001: LGTM
0002:
> > 2. This increases the memory requirements of trigram_qsort by a huge margin.
> > Could you change the radixsort to operate in-place, so that the new
> > buffer is not needed?
>
> Trigram radix sort is only called from generate_trgm() and
> generate_wildcard_trgm() which are both used to extract the unique
> trigrams from a _single_ string value.
> We don't radix sort trigrams of multiple string values. Hence, the
> maximum increase in memory, while building the GIN index, is in the size
> of the longest string encountered, not in the number of rows.
>
> Given that in-place radix sort is a lot more complex and slower, I think
> the current tradeoff is fine.
>
> Beyond that, other GIN index functions also allocate extra memory on a
> per-value basis which shows that this should not be a problem in
> practice (e.g. ginarrayextract(), gin_extract_value_trgm(), ...).
It happens, but I'd still like to avoid new O(large) allocations, if
that's possible without losing performance; allocations aren't free,
after all.
> > 3. The implementation for trigram_qsort_unsigned has not been adjusted
> > nor replaced, and so keeps the same old performance that
> > trigram_qsort_signed had.
> > Please make sure to also adjust that implementation.
>
> Done.
>
> I laid out the code such that the compiler has the possibility to fully
> inline both variants to get rid of the extra code in radix_key() for
> flipping the sign bit in the unsigned case. But even if it doesn't,
> radix_key() can be branchless and the performance is anyway dominated by
> memory traffic.
Yes, did you check that it actually gets inlined and/or optimized for
signed/unsigned versions in your local compiler?
Further note:
Now that trigram_radix_sort_with_signedness ends with a memcpy, and
every caller then calls trigram_qunique on that output, wouldn't it
make sense to include `trigram_qunique` in that final memcpy of
trigram_radix_sort?
Checking that the output is unique during the copy operation could
avoid another n_entries memory accesses vs post-copy uniqify
operations, and also save memmoves.
0003:
> > 1. ginInsertBAEntry leaks the copied key Datum if the type is by-ref
> > and already present in the accumulator.
>
> Good catch!
>
> I cannot use HASH_KEYCOPY because I'm using simplehash.h. I changed it
> to use the original key for doing the lookup and only create the copy in
> the !found path. This is also how the old code did it. Changing the key
> of an already inserted hashmap entry is save because we overwrite it
> with a copy. So the hash function will keep returning the same position
> for it.
Ah, indeed. Thanks for fixing it.
> > 2. The comment for ginInsertBAEntries was completely removed, rather
> > than updated to the new workings.
>
> Yes, because the comment no longer applies. All of it was specific to
> using a red-black tree. ginInsertBAEntries() is now merely syntactic
> sugar. I thought about removing the function completely but it's used in
> a couple of places and without it, the call sites would get slightly
> more ugly.
I see, that seems reasonable.
Newly noticed:
The new sort template for ItemPointerData should probably be a public
and reusable function, as I see many cases of qsort(..., ...,
sizeof(ItemPointerData), someItemPtrCompareFn), where qsort itself is
backed by a generic sort_template.h implementation. Pulling the
specialized implementation into its own function would allow those
callsites to be updated to the specialized version that we're
generating here.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: Reduce build times of pg_trgm GIN indexes
@ 2026-09-14 12:31 David Geier <geidav.pg@gmail.com>
parent: Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 0 replies; 51+ messages in thread
From: David Geier @ 2026-09-14 12:31 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; David Geier <geidav.pg@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; pgsql-hackers
>>> 2. This increases the memory requirements of trigram_qsort by a huge margin.
>>> Could you change the radixsort to operate in-place, so that the new
>>> buffer is not needed?
>> Trigram radix sort is only called from generate_trgm() and
>> generate_wildcard_trgm() which are both used to extract the unique
>> trigrams from a _single_ string value.
>> We don't radix sort trigrams of multiple string values. Hence, the
>> maximum increase in memory, while building the GIN index, is in the size
>> of the longest string encountered, not in the number of rows.
>>
>> Given that in-place radix sort is a lot more complex and slower, I think
>> the current tradeoff is fine.
>>
>> Beyond that, other GIN index functions also allocate extra memory on a
>> per-value basis which shows that this should not be a problem in
>> practice (e.g. ginarrayextract(), gin_extract_value_trgm(), ...).
> It happens, but I'd still like to avoid new O(large) allocations, if
> that's possible without losing performance; allocations aren't free,
> after all.
I've played around with an in-place MSD radix sort which partitions the data
and then recurses into each partition. With that code the overall
performance
regresses significantly (> 30%) which means the regression for just the sort
is even higher. I've attached my code, in case you want to take a look.
>>> 3. The implementation for trigram_qsort_unsigned has not been adjusted
>>> nor replaced, and so keeps the same old performance that
>>> trigram_qsort_signed had.
>>> Please make sure to also adjust that implementation.
>> Done.
>>
>> I laid out the code such that the compiler has the possibility to fully
>> inline both variants to get rid of the extra code in radix_key() for
>> flipping the sign bit in the unsigned case. But even if it doesn't,
>> radix_key() can be branchless and the performance is anyway dominated by
>> memory traffic.
> Yes, did you check that it actually gets inlined and/or optimized for
> signed/unsigned versions in your local compiler?
Yes, it does.
The compiler opted for putting upfront either 0 or 0x80 into the register it
later uses to XOR with, depending on if char_is_signed is true or false.
It always does the XOR but it's completely branchless inside the loops.
That's fine, given that the XOR is by no means the hotspot and duplicating
the function would increase code size.
> Further note:
> Now that trigram_radix_sort_with_signedness ends with a memcpy, and
> every caller then calls trigram_qunique on that output, wouldn't it
> make sense to include `trigram_qunique` in that final memcpy of
> trigram_radix_sort?
> Checking that the output is unique during the copy operation could
> avoid another n_entries memory accesses vs post-copy uniqify
> operations, and also save memmoves.
Good idea.
I've tried that but it's slower than doing it in-place. At least for my
benchmark which mostly processes trigram arrays of ~10 to a few thousand
trigrams
with a high number of duplicates (which is pretty realistic because in most
cases, the input strings are short to medium long).
What we could do instead is allocate a new TRGM varlena and use it as
temporary
radix sort buffer. The qunique() can then run on that buffer and we then
replace
the input TRGM with the temporary TRGM. This is marginally faster but
the code
is uglier as now the trigram radix sort function is concerned with
messing around
with the TRGM varlena. Hence, I'm leaning towards not using it.
A third variant I've tried is running qunique() on the final buffer from
the radix sort and then only copying over the unique trigrams back to
the input
buffer. The difference is within noise but I kept it because it doesn't
make the code harder to read, see attached patch.
Speaking about performance: apart from such small tweaks there are the
following
bigger optimizations I'm still planning to do, once this patch set went in:
1. Optimize the string processing code in generate_trgm_only() which
extracts
the non-unique trigrams from the input strings. Currently, this function
takes
up ~40% of all runtime.
2. Store trigrams as 32-bit integers inside the arrays to improve
qunique() and
sorting performance, as well as avoid the conversion to int-arrays in
some places.
> Newly noticed:
>
> The new sort template for ItemPointerData should probably be a public
> and reusable function, as I see many cases of qsort(..., ...,
> sizeof(ItemPointerData), someItemPtrCompareFn), where qsort itself is
> backed by a generic sort_template.h implementation. Pulling the
> specialized implementation into its own function would allow those
> callsites to be updated to the specialized version that we're
> generating here.
>
I had a look but only found four occurrences apart from the new sort in
my patch:
1. 1x in nodeTidScan.c
2. 1x in heap_surgery.c
3. 2x in test_tidstore.c
Have you come across any other?
Attached is the updated patch set. Only change is doing the qunique()
inside the
radix sort function and renaming a few function to better reflect that
apart from
sorting it now also deduplicates.
--
David Geier
Attachments:
[text/x-csrc] trigram_in_place_radix_sort.c (1.0K, ../../10afd923-38f4-451c-af89-34f8654533cb@googlemail.com/2-trigram_in_place_radix_sort.c)
download | inline:
static size_t
trigram_radix_sort_and_unique_recurse(trgm *trg, size_t count, int base,
bool char_is_signed)
{
size_t freqs[256];
size_t starts[256];
size_t cur[256];
size_t i;
int k;
if (count <= 1)
return count;
if (base >= 3)
return qunique(trg, count, sizeof(trgm), CMPTRGM_EQ);
memset(freqs, 0, sizeof(freqs));
for (i = 0; i < count; i++)
freqs[radix_key(trg[i][base], char_is_signed)]++;
starts[0] = 0;
for (k = 1; k < 256; k++)
starts[k] = starts[k - 1] + freqs[k - 1];
memcpy(cur, starts, sizeof(cur));
for (i = 0; i < count; i++)
{
for (;;)
{
unsigned char b = radix_key(trg[i][base], char_is_signed);
if (i >= starts[b] && i < cur[b])
break;
{
size_t j = cur[b]++;
trgm tmp;
memcpy(tmp, trg[i], sizeof(trgm));
memcpy(trg[i], trg[j], sizeof(trgm));
memcpy(trg[j], tmp, sizeof(trgm));
}
}
}
for (k = 0; k < 256; k++)
if (freqs[k] > 1)
trigram_radix_sort_and_unique_recurse(trg + starts[k], freqs[k], base + 1, char_is_signed);
}
[text/x-patch] v11-0003-Replace-GIN-build-accumulator-RB-tree-with-a-has.patch (17.0K, ../../10afd923-38f4-451c-af89-34f8654533cb@googlemail.com/3-v11-0003-Replace-GIN-build-accumulator-RB-tree-with-a-has.patch)
download | inline diff:
From 6d92f03ca284876d7198423432d65400717383a1 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Wed, 22 Apr 2026 14:00:40 +0200
Subject: [PATCH v11 3/3] Replace GIN build accumulator RB-tree with a hashmap
Replace the red-black tree used by ginInsertBAEntries() with a hash table
based on simplehash.h. This keeps the same basic accumulation strategy
as the previous implementation while changing key lookup and deduplication
from O(log(num_unique_keys)) tree operations to expected O(1) hash-table
operations. As a result, the overall complexity changes from
O(num_total_keys * log(num_unique_keys))
to
O(num_total_keys + num_unique_keys * log(num_unique_keys))
The latter is preferable for the usual case, where the number of unique
keys is much smaller than the number of rows. Even when most or all keys
are unique, the RB-tree rebalancing operations are sufficiently
expensive that the theoretical worst-case advantage of the tree does not
necessarily translate into better runtime.
The item pointer lists associated with each key continue to be sorted
before the accumulated entries are emitted. Use sort_template.h for this
sorting instead of qsort(). The distinct hash entries are copied into an
array and sorted using the existing GIN key comparison function so that
the output order remains unchanged.
ginInsertBAEntries() got much simpler as well. The previous implementation
inserted entries in an order intended to minimize red-black tree rebalancing
for sorted inputs. With a hash table, inserting the entries in their original
order is preferable because repeated keys are more likely to be found in the
same hash-table entry.
Use datumIsEqual() and datum_image_hash() for normal GIN keys. This is
consistent with the current GIN implementation, which does not correctly
support non-deterministic collations or types for which logically equal
values are not image-equal.
Because parallel GIN index builds also use ginInsertBAEntries(), this
change improves the accumulation phase of parallel index builds as well.
---
src/backend/access/gin/ginbulk.c | 339 +++++++++++++++----------------
src/include/access/gin_private.h | 24 +--
src/tools/pgindent/typedefs.list | 4 +-
3 files changed, 175 insertions(+), 192 deletions(-)
diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c
index 85865b39105..77c12ba932b 100644
--- a/src/backend/access/gin/ginbulk.c
+++ b/src/backend/access/gin/ginbulk.c
@@ -17,107 +17,118 @@
#include <limits.h>
#include "access/gin_private.h"
+#include "common/hashfn.h"
#include "utils/datum.h"
#include "utils/memutils.h"
+#define DEF_NENTRY 2048 /* Initial hash table size */
+#define DEF_ITEMS_PER_KEY 8 /* Initial ItemPointer array size per key */
-#define DEF_NENTRY 2048 /* GinEntryAccumulator allocation quantum */
-#define DEF_NPTR 5 /* ItemPointer initial allocation quantum */
-
-
-/* Combiner function for rbtree.c */
-static void
-ginCombineData(RBTNode *existing, const RBTNode *newdata, void *arg)
+typedef struct GinHashKey
{
- GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
- const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
+ OffsetNumber attnum;
+ GinNullCategory category;
+ Datum key;
+} GinHashKey;
- /*
- * Note this code assumes that newdata contains only one itempointer.
- */
- if (eo->count >= eo->maxcount)
- {
- if (eo->maxcount > INT_MAX)
- ereport(ERROR,
- (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("posting list is too long"),
- errhint("Reduce \"maintenance_work_mem\".")));
+typedef struct GinHashEntry
+{
+ GinHashKey hashkey;
+ uint32 hash;
+ char status;
+ ItemPointerData *items;
+ uint32 numItems;
+ uint32 allocatedItems;
+} GinHashEntry;
+
+typedef struct GinSortEntry
+{
+ GinHashKey hashkey;
+ ItemPointerData *items;
+ uint32 numItems;
+} GinSortEntry;
+
+static uint32 gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key);
+static bool gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b);
+
+#define SH_PREFIX ginbuild
+#define SH_ELEMENT_TYPE GinHashEntry
+#define SH_KEY_TYPE GinHashKey
+#define SH_KEY hashkey
+#define SH_HASH_KEY(tb, key) gin_hash_key(tb, &key)
+#define SH_EQUAL(tb, a, b) gin_equal_key(tb, &a, &b)
+#define SH_SCOPE static inline
+#define SH_STORE_HASH
+#define SH_GET_HASH(tb, a) (a)->hash
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+static uint32
+gin_hash_key(struct ginbuild_hash *tb, GinHashKey *key)
+{
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ uint32 hash;
- accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
- eo->maxcount *= 2;
- eo->list = (ItemPointerData *)
- repalloc_huge(eo->list, sizeof(ItemPointerData) * eo->maxcount);
- accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
- }
+ hash = hash_combine(0, murmurhash32((uint32) key->attnum));
+ hash = hash_combine(hash, murmurhash32((uint32) key->category));
- /* If item pointers are not ordered, they will need to be sorted later */
- if (eo->shouldSort == false)
+ if (key->category == GIN_CAT_NORM_KEY)
{
- int res;
-
- res = ginCompareItemPointers(eo->list + eo->count - 1, en->list);
- Assert(res != 0);
+ CompactAttribute *att;
- if (res > 0)
- eo->shouldSort = true;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, key->attnum - 1);
+ hash = hash_combine(hash, datum_image_hash(key->key, att->attbyval, att->attlen));
}
- eo->list[eo->count] = en->list[0];
- eo->count++;
+ return hash;
}
-/* Comparator function for rbtree.c */
-static int
-cmpEntryAccumulator(const RBTNode *a, const RBTNode *b, void *arg)
+static bool
+gin_equal_key(struct ginbuild_hash *tb, GinHashKey *a, GinHashKey *b)
{
- const GinEntryAccumulator *ea = (const GinEntryAccumulator *) a;
- const GinEntryAccumulator *eb = (const GinEntryAccumulator *) b;
- BuildAccumulator *accum = (BuildAccumulator *) arg;
-
- return ginCompareAttEntries(accum->ginstate,
- ea->attnum, ea->key, ea->category,
- eb->attnum, eb->key, eb->category);
-}
+ BuildAccumulator *accum = (BuildAccumulator *) tb->private_data;
+ CompactAttribute *att;
-/* Allocator function for rbtree.c */
-static RBTNode *
-ginAllocEntryAccumulator(void *arg)
-{
- BuildAccumulator *accum = (BuildAccumulator *) arg;
- GinEntryAccumulator *ea;
+ if (a->attnum != b->attnum)
+ return false;
+ if (a->category != b->category)
+ return false;
+ if (a->category != GIN_CAT_NORM_KEY)
+ return true;
/*
- * Allocate memory by rather big chunks to decrease overhead. We have no
- * need to reclaim RBTNodes individually, so this costs nothing.
+ * Compare the actual key values using image equality.
+ * This is correct because we don't want to deduplicate at this point.
*/
- if (accum->entryallocator == NULL || accum->eas_used >= DEF_NENTRY)
- {
- accum->entryallocator = palloc_array(GinEntryAccumulator, DEF_NENTRY);
- accum->allocatedMemory += GetMemoryChunkSpace(accum->entryallocator);
- accum->eas_used = 0;
- }
-
- /* Allocate new RBTNode from current chunk */
- ea = accum->entryallocator + accum->eas_used;
- accum->eas_used++;
-
- return (RBTNode *) ea;
+ att = TupleDescCompactAttr(accum->ginstate->origTupdesc, a->attnum - 1);
+ return datumIsEqual(a->key, b->key, att->attbyval, att->attlen);
}
+#define ST_SORT sort_itempointers
+#define ST_ELEMENT_TYPE ItemPointerData
+#define ST_COMPARE(a, b) ginCompareItemPointers(a, b)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
+#define ST_SORT sort_keys
+#define ST_ELEMENT_TYPE GinSortEntry
+#define ST_COMPARE_ARG_TYPE GinState
+#define ST_COMPARE(a, b, state) ginCompareAttEntries(state, a->hashkey.attnum, a->hashkey.key, a->hashkey.category, b->hashkey.attnum, b->hashkey.key, b->hashkey.category)
+#define ST_SCOPE static
+#define ST_DEFINE
+#include "lib/sort_template.h"
+
void
ginInitBA(BuildAccumulator *accum)
{
/* accum->ginstate is intentionally not set here */
- accum->allocatedMemory = 0;
- accum->entryallocator = NULL;
- accum->eas_used = 0;
- accum->tree = rbt_create(sizeof(GinEntryAccumulator),
- cmpEntryAccumulator,
- ginCombineData,
- ginAllocEntryAccumulator,
- NULL, /* no freefunc needed */
- accum);
+ accum->hash = ginbuild_create(CurrentMemoryContext, DEF_NENTRY, accum);
+ accum->allocatedMemory = accum->hash->size * sizeof(GinHashEntry);
+ accum->sorted_entries = NULL;
+ accum->num_entries = 0;
+ accum->current_pos = 0;
}
/*
@@ -142,124 +153,113 @@ getDatumCopy(BuildAccumulator *accum, OffsetNumber attnum, Datum value)
}
/*
- * Find/store one entry from indexed value.
+ * Insert one entry into the hash map.
+ * If the key already exists, append to its ItemPointer array.
+ * Otherwise, create a new hash entry with a new ItemPointer array.
*/
static void
ginInsertBAEntry(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum key, GinNullCategory category)
{
- GinEntryAccumulator eatmp;
- GinEntryAccumulator *ea;
- bool isNew;
+ GinHashKey hashkey;
+ GinHashEntry *entry;
+ bool found;
+ uint64 oldsize;
- /*
- * For the moment, fill only the fields of eatmp that will be looked at by
- * cmpEntryAccumulator or ginCombineData.
- */
- eatmp.attnum = attnum;
- eatmp.key = key;
- eatmp.category = category;
- /* temporarily set up single-entry itempointer list */
- eatmp.list = heapptr;
+ hashkey.attnum = attnum;
+ hashkey.category = category;
+ hashkey.key = key;
- ea = (GinEntryAccumulator *) rbt_insert(accum->tree, (RBTNode *) &eatmp,
- &isNew);
+ oldsize = accum->hash->size;
+ entry = ginbuild_insert(accum->hash, hashkey, &found);
- if (isNew)
+ if (!found)
{
/*
- * Finish initializing new tree entry, including making permanent
- * copies of the datum (if it's not null) and itempointer.
+ * Finish initializing new hashmap entry including making a permanent
+ * copy of the key.
*/
if (category == GIN_CAT_NORM_KEY)
- ea->key = getDatumCopy(accum, attnum, key);
- ea->maxcount = DEF_NPTR;
- ea->count = 1;
- ea->shouldSort = false;
- ea->list = palloc_array(ItemPointerData, DEF_NPTR);
- ea->list[0] = *heapptr;
- accum->allocatedMemory += GetMemoryChunkSpace(ea->list);
+ entry->hashkey.key = getDatumCopy(accum, attnum, key);
+
+ entry->items = palloc_array(ItemPointerData, DEF_ITEMS_PER_KEY);
+ entry->numItems = 0;
+ entry->allocatedItems = DEF_ITEMS_PER_KEY;
+ accum->allocatedMemory += (accum->hash->size - oldsize) * sizeof(GinHashEntry);
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
- else
+
+ if (entry->numItems >= entry->allocatedItems)
{
- /*
- * ginCombineData did everything needed.
- */
+ uint32 new_allocated;
+
+ if (entry->allocatedItems > UINT32_MAX / 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("too many GIN item pointers for a single key"),
+ errhint("Reduce \"maintenance_work_mem\".")));
+
+ accum->allocatedMemory -= GetMemoryChunkSpace(entry->items);
+ new_allocated = entry->allocatedItems * 2;
+ entry->items = repalloc_huge(entry->items, mul_size(sizeof(ItemPointerData), new_allocated));
+ entry->allocatedItems = new_allocated;
+ accum->allocatedMemory += GetMemoryChunkSpace(entry->items);
}
+
+ entry->items[entry->numItems++] = *heapptr;
}
-/*
- * Insert the entries for one heap pointer.
- *
- * Since the entries are being inserted into a balanced binary tree, you
- * might think that the order of insertion wouldn't be critical, but it turns
- * out that inserting the entries in sorted order results in a lot of
- * rebalancing operations and is slow. To prevent this, we attempt to insert
- * the nodes in an order that will produce a nearly-balanced tree if the input
- * is in fact sorted.
- *
- * We do this as follows. First, we imagine that we have an array whose size
- * is the smallest power of two greater than or equal to the actual array
- * size. Second, we insert the middle entry of our virtual array into the
- * tree; then, we insert the middles of each half of our virtual array, then
- * middles of quarters, etc.
- */
void
ginInsertBAEntries(BuildAccumulator *accum,
ItemPointer heapptr, OffsetNumber attnum,
Datum *entries, GinNullCategory *categories,
int32 nentries)
{
- uint32 step = nentries;
-
if (nentries <= 0)
return;
Assert(ItemPointerIsValid(heapptr) && attnum >= FirstOffsetNumber);
- /*
- * step will contain largest power of 2 and <= nentries
- */
- step |= (step >> 1);
- step |= (step >> 2);
- step |= (step >> 4);
- step |= (step >> 8);
- step |= (step >> 16);
- step >>= 1;
- step++;
-
- while (step > 0)
- {
- int i;
-
- for (i = step - 1; i < nentries && i >= 0; i += step << 1 /* *2 */ )
- ginInsertBAEntry(accum, heapptr, attnum,
- entries[i], categories[i]);
-
- step >>= 1; /* /2 */
- }
+ for (int i = 0; i < nentries; i++)
+ ginInsertBAEntry(accum, heapptr, attnum, entries[i], categories[i]);
}
-static int
-qsortCompareItemPointers(const void *a, const void *b)
-{
- int res = ginCompareItemPointers((const ItemPointerData *) a, (const ItemPointerData *) b);
-
- /* Assert that there are no equal item pointers being sorted */
- Assert(res != 0);
- return res;
-}
-
-/* Prepare to read out the rbtree contents using ginGetBAEntry */
+/* Prepare to read out the hash table contents using ginGetBAEntry */
void
ginBeginBAScan(BuildAccumulator *accum)
{
- rbt_begin_iterate(accum->tree, LeftRightWalk, &accum->tree_walk);
+ ginbuild_iterator iter;
+ GinHashEntry *entry;
+ uint32 i = 0;
+
+ accum->num_entries = accum->hash->members;
+ accum->current_pos = 0;
+
+ if (accum->num_entries == 0)
+ return;
+
+ accum->sorted_entries = palloc_array(GinSortEntry, accum->num_entries);
+ ginbuild_start_iterate(accum->hash, &iter);
+
+ while ((entry = ginbuild_iterate(accum->hash, &iter)) != NULL)
+ {
+ GinSortEntry *se = &accum->sorted_entries[i];
+ sort_itempointers(entry->items, entry->numItems);
+
+ se->hashkey = entry->hashkey;
+ se->items = entry->items;
+ se->numItems = entry->numItems;
+ i++;
+ }
+
+ Assert(i == accum->num_entries);
+ sort_keys(accum->sorted_entries, accum->num_entries, accum->ginstate);
+ accum->current_pos = 0;
}
/*
- * Get the next entry in sequence from the BuildAccumulator's rbtree.
+ * Get the next entry in sequence from the BuildAccumulator's sorted hash entries.
* This consists of a single key datum and a list (array) of one or more
* heap TIDs in which that key is found. The list is guaranteed sorted.
*/
@@ -268,25 +268,18 @@ ginGetBAEntry(BuildAccumulator *accum,
OffsetNumber *attnum, Datum *key, GinNullCategory *category,
uint32 *n)
{
- GinEntryAccumulator *entry;
- ItemPointerData *list;
-
- entry = (GinEntryAccumulator *) rbt_iterate(&accum->tree_walk);
+ GinSortEntry *entry;
- if (entry == NULL)
+ if (accum->current_pos >= accum->num_entries)
return NULL; /* no more entries */
- *attnum = entry->attnum;
- *key = entry->key;
- *category = entry->category;
- list = entry->list;
- *n = entry->count;
-
- Assert(list != NULL && entry->count > 0);
+ entry = &accum->sorted_entries[accum->current_pos];
+ accum->current_pos++;
- if (entry->shouldSort && entry->count > 1)
- qsort(list, entry->count, sizeof(ItemPointerData),
- qsortCompareItemPointers);
+ *attnum = entry->hashkey.attnum;
+ *key = entry->hashkey.key;
+ *category = entry->hashkey.category;
+ *n = entry->numItems;
- return list;
+ return entry->items;
}
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3c5fd6ba817..df6b44796c6 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -18,7 +18,6 @@
#include "common/int.h"
#include "catalog/pg_am_d.h"
#include "fmgr.h"
-#include "lib/rbtree.h"
#include "nodes/tidbitmap.h"
#include "storage/bufmgr.h"
@@ -420,26 +419,15 @@ extern void ginadjustmembers(Oid opfamilyoid,
List *functions);
/* ginbulk.c */
-typedef struct GinEntryAccumulator
-{
- RBTNode rbtnode;
- Datum key;
- GinNullCategory category;
- OffsetNumber attnum;
- bool shouldSort;
- ItemPointerData *list;
- uint32 maxcount; /* allocated size of list[] */
- uint32 count; /* current number of list[] entries */
-} GinEntryAccumulator;
typedef struct
{
- GinState *ginstate;
- Size allocatedMemory;
- GinEntryAccumulator *entryallocator;
- uint32 eas_used;
- RBTree *tree;
- RBTreeIterator tree_walk;
+ GinState * ginstate;
+ Size allocatedMemory;
+ struct ginbuild_hash * hash;
+ struct GinSortEntry * sorted_entries;
+ uint32 num_entries;
+ uint32 current_pos;
} BuildAccumulator;
extern void ginInitBA(BuildAccumulator *accum);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index de21cea65f9..e243a205b45 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1107,7 +1107,8 @@ GinBuildShared
GinBuildState
GinChkVal
GinEntries
-GinEntryAccumulator
+GinHashEntry
+GinHashKey
GinIndexStat
GinLeader
GinMetaPageData
@@ -1127,6 +1128,7 @@ GinScanKeyData
GinScanOpaque
GinScanOpaqueData
GinSegmentInfo
+GinSortEntry
GinState
GinStatsData
GinTernaryValue
--
2.53.0
[text/x-patch] v11-0002-Use-radix-sort-to-extract-trigrams.patch (4.8K, ../../10afd923-38f4-451c-af89-34f8654533cb@googlemail.com/4-v11-0002-Use-radix-sort-to-extract-trigrams.patch)
download | inline diff:
From 4a6e0875ae9b71530bc8a43ce5c25ef96d08b857 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Tue, 11 Nov 2025 13:18:59 +0100
Subject: [PATCH v11 2/3] Use radix sort to extract trigrams
Replace the comparison-based sort used by generate_trgm() and
generate_wildcard_trgm() with a three-pass radix sort.
Trigrams consist of three bytes, so their keys have a fixed and very
small width. A radix sort can therefore order them in linear time with
respect to the number of trigrams, avoiding the repeated comparator calls
and recursive partitioning performed by qsort.
The implementation preserves the existing behavior on platforms where
char is signed by flipping the most significant bit before sorting.
The radix sort requires a temporary buffer, increasing the memory
footprint while trigrams are being extracted from an input string.
However, the sort is performed separately for each string and is never
applied across multiple strings at once. Consequently, the additional
memory is limited to the processing of the current string and should
have a negligible effect on the overall memory footprint of a GIN index
build.
The resulting order remains compatible with trigram deduplication and
does not change the generated trigram sets.
---
contrib/pg_trgm/trgm_op.c | 99 ++++++++++++++++++++++++---------------
1 file changed, 61 insertions(+), 38 deletions(-)
diff --git a/contrib/pg_trgm/trgm_op.c b/contrib/pg_trgm/trgm_op.c
index 22bcc3c3361..57eaba43f14 100644
--- a/contrib/pg_trgm/trgm_op.c
+++ b/contrib/pg_trgm/trgm_op.c
@@ -226,33 +226,6 @@ CMPTRGM_CHOOSE(const void *a, const void *b)
return CMPTRGM(a, b);
}
-#define ST_SORT trigram_qsort_signed
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_SIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
-#define ST_SORT trigram_qsort_unsigned
-#define ST_ELEMENT_TYPE_VOID
-#define ST_COMPARE(a, b) CMPTRGM_UNSIGNED(a, b)
-#define ST_SCOPE static
-#define ST_DEFINE
-#define ST_DECLARE
-#include "lib/sort_template.h"
-
-/* Sort an array of trigrams, handling signedness correctly */
-static void
-trigram_qsort(trgm *array, size_t n)
-{
- if (GetDefaultCharSignedness())
- trigram_qsort_signed(array, n, sizeof(trgm));
- else
- trigram_qsort_unsigned(array, n, sizeof(trgm));
-}
-
-
/*
* Compare two trigrams for equality. This has the same signature as
* comparison functions used for sorting, so that this can be used with
@@ -268,11 +241,67 @@ CMPTRGM_EQ(const void *a, const void *b)
return aa[0] != bb[0] || aa[1] != bb[1] || aa[2] != bb[2] ? 1 : 0;
}
-/* Deduplicate an array of trigrams */
+/*
+ * Needed to properly handle negative numbers in case char is signed.
+ */
+static inline unsigned char
+radix_key(char x, bool char_is_signed)
+{
+ return char_is_signed ? x ^ 0x80 : x;
+}
+
+static inline size_t
+trigram_radix_sort_and_unique(trgm *trg, size_t count, bool char_is_signed)
+{
+ trgm *buffer = palloc_array(trgm, count);
+ trgm *starts[256];
+ trgm *from = trg;
+ trgm *to = buffer;
+ size_t freqs[256];
+
+ /*
+ * Do the sorting. Start with last character because that's the "LSB"
+ * in a trigram. Avoid unnecessary copies by ping-ponging between the buffers.
+ */
+ for (int i = 2; i >= 0; i--)
+ {
+ trgm *old_from = from;
+ trgm *next = to;
+
+ /*
+ * Compute frequencies to partition the buffer.
+ */
+ memset(freqs, 0, sizeof(freqs));
+
+ for (size_t j = 0; j < count; j++)
+ freqs[radix_key(trg[j][i], char_is_signed)]++;
+
+ for (size_t j = 0; j < 256; j++)
+ {
+ starts[j] = next;
+ next += freqs[j];
+ }
+
+ for (size_t j = 0; j < count; j++)
+ memcpy(starts[radix_key(from[j][i], char_is_signed)]++, from[j], sizeof(trgm));
+
+ from = to;
+ to = old_from;
+ }
+
+ count = qunique(buffer, count, sizeof(trgm), CMPTRGM_EQ);
+ memcpy(trg, buffer, sizeof(trgm) * count);
+ pfree(buffer);
+ return count;
+}
+
static size_t
-trigram_qunique(trgm *array, size_t n)
+trigram_sort_and_unique(trgm *array, size_t n)
{
- return qunique(array, n, sizeof(trgm), CMPTRGM_EQ);
+ if (GetDefaultCharSignedness())
+ return trigram_radix_sort_and_unique(array, n, true);
+ else
+ return trigram_radix_sort_and_unique(array, n, false);
}
/*
@@ -611,10 +640,7 @@ generate_trgm(char *str, int slen)
* Make trigrams unique.
*/
if (len > 1)
- {
- trigram_qsort(GETARR(trg), len);
- len = trigram_qunique(GETARR(trg), len);
- }
+ len = trigram_sort_and_unique(GETARR(trg), len);
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
@@ -1142,10 +1168,7 @@ generate_wildcard_trgm(const char *str, int slen)
trg = arr.datum;
len = arr.length;
if (len > 1)
- {
- trigram_qsort(GETARR(trg), len);
- len = trigram_qunique(GETARR(trg), len);
- }
+ len = trigram_sort_and_unique(GETARR(trg), len);
trg->flag = ARRKEY;
SET_VARSIZE(trg, CALCGTSIZE(ARRKEY, len));
--
2.53.0
[text/x-patch] v11-0001-Use-branchless-comparisons-in-btint4cmp-and-btin.patch (2.5K, ../../10afd923-38f4-451c-af89-34f8654533cb@googlemail.com/5-v11-0001-Use-branchless-comparisons-in-btint4cmp-and-btin.patch)
download | inline diff:
From 9f77777031665d398d7ceaad7d390630d6054911 Mon Sep 17 00:00:00 2001
From: David Geier <geidav.pg@gmail.com>
Date: Mon, 10 Nov 2025 15:40:11 +0100
Subject: [PATCH v11 1/3] Use branchless comparisons in btint4cmp and btint8cmp
Use the common pg_cmp_s32() and pg_cmp_s64() helpers to implement the
built-in B-tree comparison functions for int4 and int8.
The previous implementations used conditional branches to distinguish
less-than, equal, and greater-than values. The common comparison helpers
perform the same three-way comparison without data-dependent branches,
which can improve performance for workloads involving frequent integer
comparisons while preserving the required comparator result semantics.
btint4cmp() and btint8cmp() are PostgreSQL-callable functions invoked
through the function manager. They are not inlined at their call sites,
so replacing the original conditional implementation does not prevent a
compiler from optimizing an inline comparison in contexts where it can
see and better optimize the surrounding code. In other words, this
change affects the function-manager call path without imposing a
performance regression on callers for which the comparison could
otherwise have been inlined.
The comparison helpers also provide the appropriate handling for the
full ranges of int32 and int64 values without relying on subtraction,
which could overflow for values near the type limits.
---
src/backend/access/nbtree/nbtcompare.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 4e3a3a0f7ce..80dec200a3d 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -61,6 +61,7 @@
#include "utils/fmgrprotos.h"
#include "utils/skipsupport.h"
#include "utils/sortsupport.h"
+#include "common/int.h"
#ifdef STRESS_SORT_INT_MIN
#define A_LESS_THAN_B INT_MIN
@@ -194,12 +195,7 @@ btint4cmp(PG_FUNCTION_ARGS)
int32 a = PG_GETARG_INT32(0);
int32 b = PG_GETARG_INT32(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s32(a, b));
}
Datum
@@ -262,12 +258,7 @@ btint8cmp(PG_FUNCTION_ARGS)
int64 a = PG_GETARG_INT64(0);
int64 b = PG_GETARG_INT64(1);
- if (a > b)
- PG_RETURN_INT32(A_GREATER_THAN_B);
- else if (a == b)
- PG_RETURN_INT32(0);
- else
- PG_RETURN_INT32(A_LESS_THAN_B);
+ PG_RETURN_INT32(pg_cmp_s64(a, b));
}
Datum
--
2.53.0
^ permalink raw reply [nested|flat] 51+ messages in thread
end of thread, other threads:[~2026-09-14 12:31 UTC | newest]
Thread overview: 51+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-01-05 15:01 Reduce build times of pg_trgm GIN indexes David Geier <geidav.pg@gmail.com>
2026-01-06 17:00 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-01-06 20:05 ` Kirill Reshke <reshkekirill@gmail.com>
2026-01-09 12:10 ` David Geier <geidav.pg@gmail.com>
2026-01-09 12:06 ` David Geier <geidav.pg@gmail.com>
2026-01-09 18:36 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-01-09 21:02 ` David Geier <geidav.pg@gmail.com>
2026-01-14 12:27 ` David Geier <geidav.pg@gmail.com>
2026-01-12 22:10 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-01-21 15:45 ` David Geier <geidav.pg@gmail.com>
2026-01-21 20:50 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-01-23 10:18 ` David Geier <geidav.pg@gmail.com>
2026-03-02 12:17 ` David Geier <geidav.pg@gmail.com>
2026-03-03 17:31 ` David Geier <geidav.pg@gmail.com>
2026-04-07 11:27 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-08 02:15 ` John Naylor <johncnaylorls@gmail.com>
2026-04-13 15:05 ` David Geier <geidav.pg@gmail.com>
2026-04-14 13:05 ` David Geier <geidav.pg@gmail.com>
2026-04-09 11:28 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-04-13 09:41 ` Peter Eisentraut <peter@eisentraut.org>
2026-04-13 11:04 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-04-13 15:03 ` David Geier <geidav.pg@gmail.com>
2026-04-13 17:15 ` Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
2026-04-14 09:22 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-12 18:05 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-13 15:22 ` David Geier <geidav.pg@gmail.com>
2026-04-13 16:14 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-14 07:02 ` David Geier <geidav.pg@gmail.com>
2026-04-15 11:06 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-15 19:12 ` Peter Eisentraut <peter@eisentraut.org>
2026-04-15 21:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-16 08:45 ` Peter Eisentraut <peter@eisentraut.org>
2026-04-16 09:49 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-16 14:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-16 17:30 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-16 17:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-17 19:21 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-22 21:11 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-04-16 14:43 ` Andres Freund <andres@anarazel.de>
2026-04-13 15:06 ` David Geier <geidav.pg@gmail.com>
2026-04-14 14:24 ` David Geier <geidav.pg@gmail.com>
2026-04-22 21:26 ` David Geier <geidav.pg@gmail.com>
2026-09-01 11:05 ` David Geier <geidav.pg@gmail.com>
2026-09-01 15:21 ` Japin Li <japinli@hotmail.com>
2026-09-02 09:16 ` David Geier <geidav.pg@gmail.com>
2026-09-02 15:15 ` Japin Li <japinli@hotmail.com>
2026-09-03 07:41 ` David Geier <geidav.pg@gmail.com>
2026-09-08 14:58 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-09-09 08:49 ` David Geier <geidav.pg@gmail.com>
2026-09-09 14:30 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-09-14 12:31 ` David Geier <geidav.pg@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox