pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedSpeed up lpad() and rpad() for one-byte padding strings
12+ messages / 6 participants
[nested] [flat]
* Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 14:10 Sehrope Sarkuni <sehrope@jackdb.com>
0 siblings, 2 replies; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-23 14:10 UTC (permalink / raw)
To: pgsql-hackers
Hi hackers,
lpad() and rpad() pad one character at a time, calling
pg_mblen_range() and memcpy() once per padding char. When the padding
string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
the padding is that byte repeated, so the attached patch fills it with
one memset().
pg_mblen_range() is still called once on the byte, so a lone lead byte
of a multibyte character is rejected as before. The fast path is
skipped when no padding is needed, so such a byte is still accepted in
that case. Multibyte and multi-character padding strings still use
the loop.
One oddity of the current implementation I noticed is that the pad
string is only validated when it is being copied. So an invalid pad
string is accepted when no padding is needed. This patch preserves
that behavior but it seemed weird enough (to me) to mention here.
The existing strings.sql tests already run the one-byte path through
the default space padding. The patch adds tests to encoding.sql,
for padding with a multibyte character and for the lone-lead-byte
cases.
Timings on an AMD Ryzen 7 5700G, release build (-O3, no asserts),
pgbench -c 1 with one statement per transaction, alternating
before/after rounds, median latency in ms of 5 rounds (3 at 100M):
SELECT octet_length(rpad('x', N, ' '))
or
SELECT octet_length(lpad('x', N, '0'))
N rpad before after lpad before after
1 0.056 0.056 0.057 0.056
10 0.056 0.056 0.057 0.056
100 0.056 0.056 0.057 0.057
1000 0.062 0.056 0.062 0.056
10000 0.106 0.058 0.106 0.057
100000 0.543 0.061 0.548 0.061
1000000 4.799 0.102 4.805 0.102
10000000 53.415 5.350 53.773 5.194
100000000 567.672 96.637 567.826 96.564
controls (generic loop):
octet_length(rpad('x', 1000000, 'é')) 5.342 5.676
octet_length(lpad('x', 1000000, 'ab')) 4.841 5.078
The generic-loop controls are consistently about 5-6% slower. I
tried a few arrangements of the if-block, but they produced the same
code layout. The generic loop now straddles a 64-byte boundary where
it fit within one before. I wasn't able to eliminate that difference
by rearranging the fast path.
Large pads with a single byte seem far more common than large pads
with a multi-byte or multi-character string.
Passes check-world with asserts enabled.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
Attachments:
0001-Use-memset-for-one-byte-padding-in-lpad-and-rpad.patch (text/x-patch, 4.9K) download
From 4506127f660e28aba45ce5d69169eaaea1f1c4bb Mon Sep 17 00:00:00 2001
From: Sehrope Sarkuni <sehrope@jackdb.com>
Date: Tue, 22 Sep 2026 13:36:03 +0000
Subject: [PATCH] Use memset() for one-byte padding in lpad() and rpad()
A one-byte padding string is that byte repeated, so fill the padding
with memset() instead of the per-character loop, which called
pg_mblen_range() and memcpy() once per output byte.
pg_mblen_range() is still called once on the byte, so a lone lead byte
of a multibyte character is rejected as before. The fast path is
skipped when there is nothing to pad, so such a byte is still accepted
in that case.
---
src/backend/utils/adt/oracle_compat.c | 53 +++++++++++++++++++-------
src/test/regress/expected/encoding.out | 19 +++++++++
src/test/regress/sql/encoding.sql | 7 ++++
3 files changed, 65 insertions(+), 14 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 7422a454397..5e9459a364d 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -213,15 +213,30 @@ lpad(PG_FUNCTION_ARGS)
ptr2end = ptr2 + s2len;
ptr_ret = VARDATA(ret);
- while (m--)
+ if (s2len == 1 && m > 0)
{
- int mlen = pg_mblen_range(ptr2, ptr2end);
+ /*
+ * A one-byte padding string is a single character repeated m times,
+ * so fill it in with one memset() rather than one memcpy() per
+ * character. pg_mblen_range() is still called once so that a lone
+ * lead byte of a multibyte character is rejected as before.
+ */
+ (void) pg_mblen_range(ptr2, ptr2end);
+ memset(ptr_ret, *ptr2, m);
+ ptr_ret += m;
+ }
+ else
+ {
+ while (m--)
+ {
+ int mlen = pg_mblen_range(ptr2, ptr2end);
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
+ memcpy(ptr_ret, ptr2, mlen);
+ ptr_ret += mlen;
+ ptr2 += mlen;
+ if (ptr2 == ptr2end) /* wrap around at end of s2 */
+ ptr2 = ptr2start;
+ }
}
ptr1 = VARDATA_ANY(string1);
@@ -323,15 +338,25 @@ rpad(PG_FUNCTION_ARGS)
ptr2 = ptr2start = VARDATA_ANY(string2);
ptr2end = ptr2 + s2len;
- while (m--)
+ if (s2len == 1 && m > 0)
+ {
+ /* Same one-byte padding fast path as in lpad() */
+ (void) pg_mblen_range(ptr2, ptr2end);
+ memset(ptr_ret, *ptr2, m);
+ ptr_ret += m;
+ }
+ else
{
- int mlen = pg_mblen_range(ptr2, ptr2end);
+ while (m--)
+ {
+ int mlen = pg_mblen_range(ptr2, ptr2end);
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
+ memcpy(ptr_ret, ptr2, mlen);
+ ptr_ret += mlen;
+ ptr2 += mlen;
+ if (ptr2 == ptr2end) /* wrap around at end of s2 */
+ ptr2 = ptr2start;
+ }
}
SET_VARSIZE(ret, ptr_ret - (char *) ret);
diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out
index 0bb72a1df6f..8098599e302 100644
--- a/src/test/regress/expected/encoding.out
+++ b/src/test/regress/expected/encoding.out
@@ -60,6 +60,25 @@ SELECT reverse(good) FROM regress_encoding;
éfac
(1 row)
+-- padding with a multibyte character
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+ lpad | rpad
+---------+---------
+ ééécafé | caféééé
+(1 row)
+
+-- padding with a lone lead byte of a multibyte character = error
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT rpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+-- no error when no padding is needed
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+------+------
+ café | café
+(1 row)
+
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
ERROR: invalid byte sequence for encoding "UTF8": 0xc3
diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql
index 26caa93a5d5..7121704f790 100644
--- a/src/test/regress/sql/encoding.sql
+++ b/src/test/regress/sql/encoding.sql
@@ -37,6 +37,13 @@ SELECT substring(good, 3, 1) FROM regress_encoding;
SELECT substring(good, 4, 1) FROM regress_encoding;
SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding;
SELECT reverse(good) FROM regress_encoding;
+-- padding with a multibyte character
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+-- padding with a lone lead byte of a multibyte character = error
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT rpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+-- no error when no padding is needed
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
--
2.17.1
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 14:41 Nathan Bossart <nathandbossart@gmail.com>
parent: Sehrope Sarkuni <sehrope@jackdb.com>
1 sibling, 2 replies; 12+ messages in thread
From: Nathan Bossart @ 2026-09-23 14:41 UTC (permalink / raw)
To: Sehrope Sarkuni <sehrope@jackdb.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
> lpad() and rpad() pad one character at a time, calling
> pg_mblen_range() and memcpy() once per padding char. When the padding
> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
> the padding is that byte repeated, so the attached patch fills it with
> one memset().
I wonder if we could expand these gains by using SIMD whenever the vector
length is divisible by the padding string length. My hunch is that's where
a lot of the memset() gains come from.
--
nathan
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 17:09 Nathan Bossart <nathandbossart@gmail.com>
parent: Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 12+ messages in thread
From: Nathan Bossart @ 2026-09-23 17:09 UTC (permalink / raw)
To: Sehrope Sarkuni <sehrope@jackdb.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 09:41:36AM -0500, Nathan Bossart wrote:
> On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
>> lpad() and rpad() pad one character at a time, calling
>> pg_mblen_range() and memcpy() once per padding char. When the padding
>> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
>> the padding is that byte repeated, so the attached patch fills it with
>> one memset().
>
> I wonder if we could expand these gains by using SIMD whenever the vector
> length is divisible by the padding string length. My hunch is that's where
> a lot of the memset() gains come from.
Actually, I think we can expand this to any padding string length by
copying the padding string once, and then copying from the beginning of the
padding to the end repeatedly so that we write double the padding each
time. This is a bit like what commit c60e520 added for pglz_decompress().
I've attached some proof-of-concept grade patches. This doesn't quite
match the performance of your 1-byte fast-path, but it's pretty close and
applies to many more cases.
--
nathan
Attachments: download all (2 files, .tar.gz)
v2-0001-Factor-the-padding-loop-out-of-lpad-and-rpad.patch (text/plain, 2.9K) download
From db26e9d41843c6837f3c4ce77c4667bac5c00644 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 23 Sep 2026 11:47:02 -0500
Subject: [PATCH v2 1/2] Factor the padding loop out of lpad() and rpad().
lpad() and rpad() each carry an identical copy of the loop that
writes the padding characters. This commit moves it into a helper
function that both call. No functional change.
This is preparatory work for a follow-up commit that will teach the
helper to write long padding with a few large memcpy() calls instead
of one per character.
---
src/backend/utils/adt/oracle_compat.c | 60 ++++++++++++---------------
1 file changed, 27 insertions(+), 33 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 7422a454397..2adc3874bc2 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -143,6 +143,31 @@ casefold(PG_FUNCTION_ARGS)
}
+/*
+ * Append m characters of the padding string s2 (s2len bytes) to dst,
+ * cycling through s2 as needed, and return a pointer past the last byte
+ * written.
+ */
+static char *
+append_padding(char *dst, const char *s2, int s2len, int m)
+{
+ const char *ptr2 = s2;
+ const char *ptr2end = s2 + s2len;
+
+ while (m--)
+ {
+ int mlen = pg_mblen_range(ptr2, ptr2end);
+
+ memcpy(dst, ptr2, mlen);
+ dst += mlen;
+ ptr2 += mlen;
+ if (ptr2 == ptr2end) /* wrap around at end of s2 */
+ ptr2 = s2;
+ }
+
+ return dst;
+}
+
/********************************************************************
*
* lpad
@@ -167,10 +192,7 @@ lpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -209,20 +231,7 @@ lpad(PG_FUNCTION_ARGS)
m = len - s1len;
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
- ptr_ret = VARDATA(ret);
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(VARDATA(ret), VARDATA_ANY(string2), s2len, m);
ptr1 = VARDATA_ANY(string1);
@@ -265,10 +274,7 @@ rpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -320,19 +326,7 @@ rpad(PG_FUNCTION_ARGS)
ptr1 += mlen;
}
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(ptr_ret, VARDATA_ANY(string2), s2len, m);
SET_VARSIZE(ret, ptr_ret - (char *) ret);
--
2.55.0
v2-0002-Optimize-padding-in-lpad-and-rpad.patch (text/plain, 5.3K) download
From 8bdb4237a3b08b192ddfa7d69128bff3e0b92d47 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 23 Sep 2026 11:47:23 -0500
Subject: [PATCH v2 2/2] Optimize padding in lpad() and rpad().
lpad() and rpad() write the padding one character at a time, calling
pg_mblen_range() and memcpy() for each one, which gets slow once the
padding runs to many kilobytes. Since the padding is just the pad
string repeated, this commit copies the pad string once, as before,
and then produces the rest by copying what has already been written
onto the end of itself, doubling the length each time. That takes a
handful of memcpy() calls for a pad string of any length, and it is
dramatically faster for long padding.
Note that the pad string is still only checked for a truncated
multibyte character as far as it is actually used, so a bad tail is
accepted when no padding is needed, as before.
Co-authored-by: Sehrope Sarkuni <sehrope@jackdb.com>
Discussion: https://postgr.es/m/CAH7T-apj%2BpFg9bRkXVGfGejS2Uu4WzdMd7KoLKTW11mdsrt1Ew%40mail.gmail.com
---
src/backend/utils/adt/oracle_compat.c | 45 ++++++++++++++++++++++++--
src/test/regress/expected/encoding.out | 19 +++++++++++
src/test/regress/sql/encoding.sql | 7 ++++
3 files changed, 68 insertions(+), 3 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 2adc3874bc2..d54fe948355 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -147,22 +147,61 @@ casefold(PG_FUNCTION_ARGS)
* Append m characters of the padding string s2 (s2len bytes) to dst,
* cycling through s2 as needed, and return a pointer past the last byte
* written.
+ *
+ * Note that s2 is only checked for a truncated multibyte character as far as
+ * we actually use it, so a bad tail is accepted when the padding stops short
+ * of it, as it always has been.
*/
static char *
append_padding(char *dst, const char *s2, int s2len, int m)
{
const char *ptr2 = s2;
const char *ptr2end = s2 + s2len;
+ char *start = dst;
+ int nchars = 0;
+ int nbytes;
+ int written;
- while (m--)
+ /* Copy s2 once, one character at a time, or until m runs out. */
+ while (m > 0 && ptr2 < ptr2end)
{
int mlen = pg_mblen_range(ptr2, ptr2end);
memcpy(dst, ptr2, mlen);
dst += mlen;
ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = s2;
+ m--;
+ nchars++;
+ }
+
+ if (m == 0)
+ return dst;
+
+ /*
+ * The rest of the padding is s2 repeated, so work out how many bytes that
+ * is: whole copies of s2, plus the first m % nchars characters of one
+ * more.
+ */
+ nbytes = (m / nchars) * s2len;
+ ptr2 = s2;
+ for (m %= nchars; m > 0; m--)
+ ptr2 += pg_mblen_unbounded(ptr2);
+ nbytes += ptr2 - s2;
+
+ /*
+ * Now produce those bytes by copying what we've already written, doubling
+ * the length each time, so that the work is done by a few large memcpy()
+ * calls rather than one per character.
+ */
+ written = dst - start;
+ while (nbytes > 0)
+ {
+ int chunk = Min(written, nbytes);
+
+ memcpy(dst, start, chunk);
+ dst += chunk;
+ nbytes -= chunk;
+ written += chunk;
}
return dst;
diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out
index 0bb72a1df6f..8098599e302 100644
--- a/src/test/regress/expected/encoding.out
+++ b/src/test/regress/expected/encoding.out
@@ -60,6 +60,25 @@ SELECT reverse(good) FROM regress_encoding;
éfac
(1 row)
+-- padding with a multibyte character
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+ lpad | rpad
+---------+---------
+ ééécafé | caféééé
+(1 row)
+
+-- padding with a lone lead byte of a multibyte character = error
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT rpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+-- no error when no padding is needed
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+------+------
+ café | café
+(1 row)
+
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
ERROR: invalid byte sequence for encoding "UTF8": 0xc3
diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql
index 26caa93a5d5..7121704f790 100644
--- a/src/test/regress/sql/encoding.sql
+++ b/src/test/regress/sql/encoding.sql
@@ -37,6 +37,13 @@ SELECT substring(good, 3, 1) FROM regress_encoding;
SELECT substring(good, 4, 1) FROM regress_encoding;
SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding;
SELECT reverse(good) FROM regress_encoding;
+-- padding with a multibyte character
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+-- padding with a lone lead byte of a multibyte character = error
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT rpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+-- no error when no padding is needed
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
--
2.55.0
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 17:16 Sehrope Sarkuni <sehrope@jackdb.com>
parent: Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 0 replies; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-23 17:16 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 10:41 AM Nathan Bossart
<nathandbossart@gmail.com> wrote:
>
> On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
> > lpad() and rpad() pad one character at a time, calling
> > pg_mblen_range() and memcpy() once per padding char. When the padding
> > string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
> > the padding is that byte repeated, so the attached patch fills it with
> > one memset().
>
> I wonder if we could expand these gains by using SIMD whenever the vector
> length is divisible by the padding string length. My hunch is that's where
> a lot of the memset() gains come from.
I think the repeated pg_mblen_range() calls for every iteration are a
big factor too. I tried a different route that covers every pad
string (not just divisible) without explicit SIMD.
The attached v2 walks the pad string once to count its characters,
copies the whole repetitions as byte sequences, and runs the
per-character loop only for the partial final repetition. The whole
repetitions are written by copying one and then doubling the copied
region, so it takes log2(repetitions) memcpy() calls and memcpy()
does the vectorizing.
Validation is unchanged from master. The counting pass stops at the
number of characters needed, so a pad string ending in a lone lead
byte still errors only if the padding reaches that byte. The two
identical loops in lpad() and rpad() become a static pad_fill().
This replaces the memset() patch. On the one-byte case memset() was
5-7% faster than doubling (0.100 vs 0.107 ms at 1M), which does not
seem worth a separate fast path, and the multi-character controls that
were 5% slower with v1 now get the same speedup as everything else.
Same setup as before, median ms of 5 rounds:
SELECT octet_length(rpad('x', N, PAD))
PAD N before after
' ' 1000000 4.852 0.108
'ab' 1000000 4.822 0.111
'abc' 999999 4.824 0.107 (exact multiple)
'abc' 1000000 4.793 0.108 (partial tail)
'abcd' 1000000 4.809 0.110
16 chars 1000000 5.012 0.107
100 chars 1000000 4.917 0.110
'é' 1000000 5.394 0.155
'aéb' 999999 5.012 0.126 (exact multiple)
'aéb' 1000000 4.994 0.122 (partial tail)
' ' 10000000 53.705 5.466
'ab' 10000000 53.376 5.413
'abc' 1000 0.063 0.057
'abc' 10 0.056 0.056
This is more of a change than just adding the memset() fastpath, but I
think the end result of the code is easier to follow too with both
lpad() and rpad() sharing the helper.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
Attachments:
v2-0001-Copy-whole-repetitions-of-the-pad-string-at-once-.patch (text/x-patch, 9.0K) download
From 512b8c05da4d7db8108a3ea9234f09307d35ed15 Mon Sep 17 00:00:00 2001
From: Sehrope Sarkuni <sehrope@jackdb.com>
Date: Wed, 23 Sep 2026 15:29:23 +0000
Subject: [PATCH v2] Copy whole repetitions of the pad string at once in lpad()
and rpad()
The padding loop called pg_mblen_range() and memcpy() once per
character. Walk the pad string once to count its characters, copy the
whole repetitions as byte sequences by doubling the copied region, and
use the per-character loop only for the partial final repetition.
The pad string is validated as far as before, which is all of it when
a full repetition is copied and otherwise only the characters copied.
---
src/backend/utils/adt/oracle_compat.c | 101 +++++++++++++++++--------
src/test/regress/expected/encoding.out | 37 +++++++++
src/test/regress/expected/strings.out | 26 +++++++
src/test/regress/sql/encoding.sql | 10 +++
src/test/regress/sql/strings.sql | 7 ++
5 files changed, 150 insertions(+), 31 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 7422a454397..c2c58f28e38 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -143,6 +143,74 @@ casefold(PG_FUNCTION_ARGS)
}
+/*
+ * Write m characters of padding at dst, taken cyclically from the pad
+ * string of padlen bytes, and return the number of bytes written.
+ *
+ * The pad string is validated with pg_mblen_range() only as far as it is
+ * used, so an incomplete multibyte character at its end is an error only
+ * if the padding reaches it.
+ */
+static int
+pad_fill(char *dst, const char *pad, int padlen, int m)
+{
+ const char *padend = pad + padlen;
+ const char *p = pad;
+ int nchars = 0;
+ int nrep;
+ int total;
+ int copied;
+
+ if (m <= 0)
+ return 0;
+
+ /* count the characters of one repetition, stopping at m */
+ while (p < padend && nchars < m)
+ {
+ p += pg_mblen_range(p, padend);
+ nchars++;
+ }
+
+ /* fewer than one repetition is needed, so copy the first m characters */
+ if (p < padend)
+ {
+ memcpy(dst, pad, p - pad);
+ return p - pad;
+ }
+
+ /*
+ * Whole repetitions are byte copies of the pad string. Copy one, then
+ * double the copied region until all of them are written, so the number
+ * of memcpy() calls is logarithmic in the number of repetitions. total
+ * cannot overflow, since the caller sized the output for m characters.
+ */
+ nrep = m / nchars;
+ total = nrep * padlen;
+ memcpy(dst, pad, padlen);
+ copied = padlen;
+ while (copied < total)
+ {
+ int n = Min(copied, total - copied);
+
+ memcpy(dst + copied, dst, n);
+ copied += n;
+ }
+
+ /* partial final repetition, one character at a time */
+ m -= nrep * nchars;
+ p = pad;
+ while (m-- > 0)
+ {
+ int mlen = pg_mblen_range(p, padend);
+
+ memcpy(dst + copied, p, mlen);
+ copied += mlen;
+ p += mlen;
+ }
+
+ return copied;
+}
+
/********************************************************************
*
* lpad
@@ -167,10 +235,7 @@ lpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -209,20 +274,9 @@ lpad(PG_FUNCTION_ARGS)
m = len - s1len;
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
ptr_ret = VARDATA(ret);
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret += pad_fill(ptr_ret, VARDATA_ANY(string2), s2len, m);
ptr1 = VARDATA_ANY(string1);
@@ -265,10 +319,7 @@ rpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -320,19 +371,7 @@ rpad(PG_FUNCTION_ARGS)
ptr1 += mlen;
}
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret += pad_fill(ptr_ret, VARDATA_ANY(string2), s2len, m);
SET_VARSIZE(ret, ptr_ret - (char *) ret);
diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out
index 0bb72a1df6f..9fc871215b0 100644
--- a/src/test/regress/expected/encoding.out
+++ b/src/test/regress/expected/encoding.out
@@ -60,6 +60,43 @@ SELECT reverse(good) FROM regress_encoding;
éfac
(1 row)
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+ lpad | rpad
+---------+---------
+ ééécafé | caféééé
+(1 row)
+
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+ lpad | rpad
+--------------+--------------
+ éabéabéacafé | cafééabéabéa
+(1 row)
+
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+ lpad | rpad
+-------+-------
+ écafé | caféé
+(1 row)
+
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+-------+--------
+ acafé | caféab
+(1 row)
+
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+------+------
+ café | café
+(1 row)
+
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
ERROR: invalid byte sequence for encoding "UTF8": 0xc3
diff --git a/src/test/regress/expected/strings.out b/src/test/regress/expected/strings.out
index fa29abfd829..6e064e3807f 100644
--- a/src/test/regress/expected/strings.out
+++ b/src/test/regress/expected/strings.out
@@ -3441,6 +3441,32 @@ SELECT rpad('hi', 5, '');
hi
(1 row)
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+ lpad | rpad
+----------+----------
+ abcabchi | hiabcabc
+(1 row)
+
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+ lpad | rpad
+-----------+-----------
+ abcabcahi | hiabcabca
+(1 row)
+
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+ lpad | rpad
+--------------+--------------
+ abababababhi | hiababababab
+(1 row)
+
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+ lpad | rpad
+------+------
+ ahi | hia
+(1 row)
+
SELECT ltrim('zzzytrim', 'xyz');
ltrim
-------
diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql
index 26caa93a5d5..3ea6e54e52d 100644
--- a/src/test/regress/sql/encoding.sql
+++ b/src/test/regress/sql/encoding.sql
@@ -37,6 +37,16 @@ SELECT substring(good, 3, 1) FROM regress_encoding;
SELECT substring(good, 4, 1) FROM regress_encoding;
SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding;
SELECT reverse(good) FROM regress_encoding;
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
diff --git a/src/test/regress/sql/strings.sql b/src/test/regress/sql/strings.sql
index 7d9c7275a02..04651a0a46f 100644
--- a/src/test/regress/sql/strings.sql
+++ b/src/test/regress/sql/strings.sql
@@ -1165,6 +1165,13 @@ SELECT rpad('hi', -5, 'xy');
SELECT rpad('hello', 2);
SELECT rpad('hi', 5, '');
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+
SELECT ltrim('zzzytrim', 'xyz');
SELECT translate('', '14', 'ax');
--
2.43.0
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 17:19 Sehrope Sarkuni <sehrope@jackdb.com>
parent: Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 2 replies; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-23 17:19 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 1:09 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Wed, Sep 23, 2026 at 09:41:36AM -0500, Nathan Bossart wrote:
> > On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
> >> lpad() and rpad() pad one character at a time, calling
> >> pg_mblen_range() and memcpy() once per padding char. When the padding
> >> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
> >> the padding is that byte repeated, so the attached patch fills it with
> >> one memset().
> >
> > I wonder if we could expand these gains by using SIMD whenever the vector
> > length is divisible by the padding string length. My hunch is that's where
> > a lot of the memset() gains come from.
>
> Actually, I think we can expand this to any padding string length by
> copying the padding string once, and then copying from the beginning of the
> padding to the end repeatedly so that we write double the padding each
> time. This is a bit like what commit c60e520 added for pglz_decompress().
> I've attached some proof-of-concept grade patches. This doesn't quite
> match the performance of your 1-byte fast-path, but it's pretty close and
> applies to many more cases.
Ah! just saw this after I hit send.
I think my version does the rest of what you're describing!
Exact conclusion reached from testing it too.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-23 22:51 Sehrope Sarkuni <sehrope@jackdb.com>
parent: Sehrope Sarkuni <sehrope@jackdb.com>
1 sibling, 0 replies; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-23 22:51 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 1:19 PM Sehrope Sarkuni <sehrope@jackdb.com> wrote:
>
> On Wed, Sep 23, 2026 at 1:09 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> >
> > On Wed, Sep 23, 2026 at 09:41:36AM -0500, Nathan Bossart wrote:
> > > On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
> > >> lpad() and rpad() pad one character at a time, calling
> > >> pg_mblen_range() and memcpy() once per padding char. When the padding
> > >> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
> > >> the padding is that byte repeated, so the attached patch fills it with
> > >> one memset().
> > >
> > > I wonder if we could expand these gains by using SIMD whenever the vector
> > > length is divisible by the padding string length. My hunch is that's where
> > > a lot of the memset() gains come from.
> >
> > Actually, I think we can expand this to any padding string length by
> > copying the padding string once, and then copying from the beginning of the
> > padding to the end repeatedly so that we write double the padding each
> > time. This is a bit like what commit c60e520 added for pglz_decompress().
> > I've attached some proof-of-concept grade patches. This doesn't quite
> > match the performance of your 1-byte fast-path, but it's pretty close and
> > applies to many more cases.
>
> Ah! just saw this after I hit send.
I like your split with the refactor into the separate function first
better. It makes the second piece purely the perf improvement. And
the copy-as-you-count for the first piece is nifty.
Attached is v4 (retroactively referring to my v2, as v3). I stepped
through and renamed it a bit. The rest of it is your v2 with some
updated tests.
Passes tests, CI, and the numbers match my v3.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
Attachments: download all (2 files, .tar.gz)
v4-0001-Factor-the-padding-loop-out-of-lpad-and-rpad.patch (text/x-patch, 2.9K) download
From d8a3b9507845c28c2e458cd57637524e1eacfc95 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 23 Sep 2026 11:47:02 -0500
Subject: [PATCH v4 1/2] Factor the padding loop out of lpad() and rpad().
lpad() and rpad() each carry an identical copy of the loop that
writes the padding characters. This commit moves it into a helper
function that both call. No functional change.
This is preparatory work for a follow-up commit that will teach the
helper to write long padding with a few large memcpy() calls instead
of one per character.
---
src/backend/utils/adt/oracle_compat.c | 60 ++++++++++++---------------
1 file changed, 27 insertions(+), 33 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 7422a454397..e5238e44813 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -143,6 +143,31 @@ casefold(PG_FUNCTION_ARGS)
}
+/*
+ * Append m characters of the padding string pad (padlen bytes) to dst,
+ * cycling through pad as needed, and return a pointer past the last byte
+ * written.
+ */
+static char *
+append_padding(char *dst, const char *pad, int padlen, int m)
+{
+ const char *p = pad;
+ const char *pend = pad + padlen;
+
+ while (m--)
+ {
+ int mlen = pg_mblen_range(p, pend);
+
+ memcpy(dst, p, mlen);
+ dst += mlen;
+ p += mlen;
+ if (p == pend) /* wrap around at end of pad */
+ p = pad;
+ }
+
+ return dst;
+}
+
/********************************************************************
*
* lpad
@@ -167,10 +192,7 @@ lpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -209,20 +231,7 @@ lpad(PG_FUNCTION_ARGS)
m = len - s1len;
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
- ptr_ret = VARDATA(ret);
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(VARDATA(ret), VARDATA_ANY(string2), s2len, m);
ptr1 = VARDATA_ANY(string1);
@@ -265,10 +274,7 @@ rpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -320,19 +326,7 @@ rpad(PG_FUNCTION_ARGS)
ptr1 += mlen;
}
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(ptr_ret, VARDATA_ANY(string2), s2len, m);
SET_VARSIZE(ret, ptr_ret - (char *) ret);
--
2.17.1
v4-0002-Optimize-padding-in-lpad-and-rpad.patch (text/x-patch, 7.9K) download
From a5007ff32afe3bd608a88d0f64a79bdb327e8448 Mon Sep 17 00:00:00 2001
From: Sehrope Sarkuni <sehrope@jackdb.com>
Date: Wed, 23 Sep 2026 21:41:57 +0000
Subject: [PATCH v4 2/2] Optimize padding in lpad() and rpad().
lpad() and rpad() write the padding one character at a time, calling
pg_mblen_range() and memcpy() for each one, which gets slow once the
padding runs to many kilobytes. Since the padding is just the pad
string repeated, this commit copies the pad string once, as before,
and then produces the rest by copying what has already been written
onto the end of itself, doubling the length each time. That takes a
handful of memcpy() calls for a pad string of any length, and it is
dramatically faster for long padding.
Note that the pad string is still only checked for a truncated
multibyte character as far as it is actually used, so a bad tail is
accepted when no padding is needed, as before.
---
src/backend/utils/adt/oracle_compat.c | 47 ++++++++++++++++++++++++--
src/test/regress/expected/encoding.out | 37 ++++++++++++++++++++
src/test/regress/expected/strings.out | 26 ++++++++++++++
src/test/regress/sql/encoding.sql | 10 ++++++
src/test/regress/sql/strings.sql | 7 ++++
5 files changed, 124 insertions(+), 3 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index e5238e44813..f457e01a2ae 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -147,22 +147,63 @@ casefold(PG_FUNCTION_ARGS)
* Append m characters of the padding string pad (padlen bytes) to dst,
* cycling through pad as needed, and return a pointer past the last byte
* written.
+ *
+ * The pad string is validated with pg_mblen_range() only as far as it is
+ * used, so an incomplete multibyte character at its end is an error only
+ * if the padding reaches it.
*/
static char *
append_padding(char *dst, const char *pad, int padlen, int m)
{
const char *p = pad;
const char *pend = pad + padlen;
+ char *start = dst;
+ int nchars = 0;
+ int nbytes;
+ int written;
- while (m--)
+ /* copy pad once, one character at a time, or until m runs out */
+ while (m > 0 && p < pend)
{
int mlen = pg_mblen_range(p, pend);
memcpy(dst, p, mlen);
dst += mlen;
p += mlen;
- if (p == pend) /* wrap around at end of pad */
- p = pad;
+ m--;
+ nchars++;
+ }
+
+ if (m == 0)
+ return dst;
+
+ /*
+ * The rest of the padding is pad repeated, so work out how many bytes
+ * that is: whole copies of pad, plus the first m % nchars characters of
+ * one more.
+ */
+ nbytes = (m / nchars) * padlen;
+ p = pad;
+ for (m %= nchars; m > 0; m--)
+ p += pg_mblen_unbounded(p);
+ nbytes += p - pad;
+
+ /*
+ * Produce those bytes by copying what has already been written onto the
+ * end, doubling the length each time, so the work is done by a few large
+ * memcpy() calls rather than one per character. The last chunk, if
+ * shorter, is a prefix of the padding written so far and therefore of
+ * pad, which is the partial final repetition.
+ */
+ written = dst - start;
+ while (nbytes > 0)
+ {
+ int chunk = Min(written, nbytes);
+
+ memcpy(dst, start, chunk);
+ dst += chunk;
+ nbytes -= chunk;
+ written += chunk;
}
return dst;
diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out
index 0bb72a1df6f..9fc871215b0 100644
--- a/src/test/regress/expected/encoding.out
+++ b/src/test/regress/expected/encoding.out
@@ -60,6 +60,43 @@ SELECT reverse(good) FROM regress_encoding;
éfac
(1 row)
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+ lpad | rpad
+---------+---------
+ ééécafé | caféééé
+(1 row)
+
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+ lpad | rpad
+--------------+--------------
+ éabéabéacafé | cafééabéabéa
+(1 row)
+
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+ lpad | rpad
+-------+-------
+ écafé | caféé
+(1 row)
+
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+-------+--------
+ acafé | caféab
+(1 row)
+
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+------+------
+ café | café
+(1 row)
+
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
ERROR: invalid byte sequence for encoding "UTF8": 0xc3
diff --git a/src/test/regress/expected/strings.out b/src/test/regress/expected/strings.out
index fa29abfd829..6e064e3807f 100644
--- a/src/test/regress/expected/strings.out
+++ b/src/test/regress/expected/strings.out
@@ -3441,6 +3441,32 @@ SELECT rpad('hi', 5, '');
hi
(1 row)
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+ lpad | rpad
+----------+----------
+ abcabchi | hiabcabc
+(1 row)
+
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+ lpad | rpad
+-----------+-----------
+ abcabcahi | hiabcabca
+(1 row)
+
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+ lpad | rpad
+--------------+--------------
+ abababababhi | hiababababab
+(1 row)
+
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+ lpad | rpad
+------+------
+ ahi | hia
+(1 row)
+
SELECT ltrim('zzzytrim', 'xyz');
ltrim
-------
diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql
index 26caa93a5d5..3ea6e54e52d 100644
--- a/src/test/regress/sql/encoding.sql
+++ b/src/test/regress/sql/encoding.sql
@@ -37,6 +37,16 @@ SELECT substring(good, 3, 1) FROM regress_encoding;
SELECT substring(good, 4, 1) FROM regress_encoding;
SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding;
SELECT reverse(good) FROM regress_encoding;
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
diff --git a/src/test/regress/sql/strings.sql b/src/test/regress/sql/strings.sql
index 7d9c7275a02..04651a0a46f 100644
--- a/src/test/regress/sql/strings.sql
+++ b/src/test/regress/sql/strings.sql
@@ -1165,6 +1165,13 @@ SELECT rpad('hi', -5, 'xy');
SELECT rpad('hello', 2);
SELECT rpad('hi', 5, '');
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+
SELECT ltrim('zzzytrim', 'xyz');
SELECT translate('', '14', 'ax');
--
2.17.1
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-24 05:21 John Naylor <johncnaylorls@gmail.com>
parent: Sehrope Sarkuni <sehrope@jackdb.com>
1 sibling, 1 reply; 12+ messages in thread
From: John Naylor @ 2026-09-24 05:21 UTC (permalink / raw)
To: Sehrope Sarkuni <sehrope@jackdb.com>; +Cc: pgsql-hackers
On Wed, Sep 23, 2026 at 9:10 PM Sehrope Sarkuni <sehrope@jackdb.com> wrote:
> Timings on an AMD Ryzen 7 5700G, release build (-O3, no asserts),
> pgbench -c 1 with one statement per transaction, alternating
> before/after rounds, median latency in ms of 5 rounds (3 at 100M):
>
> SELECT octet_length(rpad('x', N, ' '))
> or
> SELECT octet_length(lpad('x', N, '0'))
>
> N rpad before after lpad before after
> 1 0.056 0.056 0.057 0.056
> 10 0.056 0.056 0.057 0.056
> 100 0.056 0.056 0.057 0.057
100 is already far beyond the usual N values I've seen in real life,
and there is no measured difference here, so I don't see what problem
this is solving.
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-24 11:03 Sehrope Sarkuni <sehrope@jackdb.com>
parent: John Naylor <johncnaylorls@gmail.com>
0 siblings, 0 replies; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-24 11:03 UTC (permalink / raw)
To: John Naylor <johncnaylorls@gmail.com>; +Cc: pgsql-hackers
On Thu, Sep 24, 2026 at 1:21 AM John Naylor <johncnaylorls@gmail.com> wrote:
>
> On Wed, Sep 23, 2026 at 9:10 PM Sehrope Sarkuni <sehrope@jackdb.com> wrote:
> > Timings on an AMD Ryzen 7 5700G, release build (-O3, no asserts),
> > pgbench -c 1 with one statement per transaction, alternating
> > before/after rounds, median latency in ms of 5 rounds (3 at 100M):
> >
> > SELECT octet_length(rpad('x', N, ' '))
> > or
> > SELECT octet_length(lpad('x', N, '0'))
> >
> > N rpad before after lpad before after
> > 1 0.056 0.056 0.057 0.056
> > 10 0.056 0.056 0.057 0.056
> > 100 0.056 0.056 0.057 0.057
>
> 100 is already far beyond the usual N values I've seen in real life,
> and there is no measured difference here, so I don't see what problem
> this is solving.
That's probably true for the usual formatting uses.
But large N does show up for test data. We already have this in
src/test/regress/sql/join_hash.sql:
--- Make a relation with several enormous tuples.
create table wide as select generate_series(1, 3) as id, rpad('',
320000, 'x') as t;
There are similar uses in benchmarks and reproducers in the wild too.
So I don't think large N is particularly artificial. We also have lots
of places in the tree doing essentially the same thing with
'foo' || repeat('bar', large_N) to generate test data.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-24 11:20 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Sehrope Sarkuni <sehrope@jackdb.com>
1 sibling, 1 reply; 12+ messages in thread
From: Heikki Linnakangas @ 2026-09-24 11:20 UTC (permalink / raw)
To: Sehrope Sarkuni <sehrope@jackdb.com>; Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers
On 23/09/2026 20:19, Sehrope Sarkuni wrote:
> On Wed, Sep 23, 2026 at 1:09 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>>
>> On Wed, Sep 23, 2026 at 09:41:36AM -0500, Nathan Bossart wrote:
>>> On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote:
>>>> lpad() and rpad() pad one character at a time, calling
>>>> pg_mblen_range() and memcpy() once per padding char. When the padding
>>>> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '),
>>>> the padding is that byte repeated, so the attached patch fills it with
>>>> one memset().
>>>
>>> I wonder if we could expand these gains by using SIMD whenever the vector
>>> length is divisible by the padding string length. My hunch is that's where
>>> a lot of the memset() gains come from.
>>
>> Actually, I think we can expand this to any padding string length by
>> copying the padding string once, and then copying from the beginning of the
>> padding to the end repeatedly so that we write double the padding each
>> time. This is a bit like what commit c60e520 added for pglz_decompress().
>> I've attached some proof-of-concept grade patches. This doesn't quite
>> match the performance of your 1-byte fast-path, but it's pretty close and
>> applies to many more cases.
>
> Ah! just saw this after I hit send.
>
> I think my version does the rest of what you're describing!
>
> Exact conclusion reached from testing it too.
See also similar thread on REPEAT():
https://www.postgresql.org/message-id/tencent_C5BBECF985A270FBC49463EDAF722CD5E005%40qq.com.
Whatever we do here, let's use the same implementation for REPEAT(),
LPAD(), and RPAD().
- Heikki
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-24 23:54 Sehrope Sarkuni <sehrope@jackdb.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 12+ messages in thread
From: Sehrope Sarkuni @ 2026-09-24 23:54 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Nathan Bossart <nathandbossart@gmail.com>; pgsql-hackers
On Thu, Sep 24, 2026 at 7:20 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> See also similar thread on REPEAT():
> https://www.postgresql.org/message-id/tencent_C5BBECF985A270FBC49463EDAF722CD5E005%40qq.com.
Thanks for linking to that. A lot overlap in ideas and results too.
> Whatever we do here, let's use the same implementation for REPEAT(),
> LPAD(), and RPAD().
Attached v5 moves the doubling copy into its own helper so repeat()
can use it too.
0001 is the refactor for rpad/lpad to use append_padding().
0002 adds repeat_bytes(dst, src, srclen, count) and rebuilds
append_padding() on it.
0003 replaces repeat()'s per-copy loop with a call to repeat_bytes().
repeat_bytes() copies the string once, doubles the copied region
until it is at least 16 kB, and then copies that region repeatedly
with a CHECK_FOR_INTERRUPTS() per copy. Pure doubling in v4 was 6-8%
slower than master's loop for repeat() with a source of 100 bytes or
more and a 100 MB result, because the loop's source stays in L1 while
the doubling reads back what it just wrote.
The fixed block keeps the source in cache and puts those cases back
at parity, while still needing only a few calls for short strings.
It also keeps large results cancellable. With and without the
interrupts check was not measurable.
Timings on an AMD Ryzen 7 5700G, release build (-O3, no asserts),
pgbench -c 1, alternating before/after rounds, median ms of 5 rounds.
Pad is 'abcdefghij' repeated and cut to len. N is the result size in
bytes. Each cell is master / patched.
# repeat(pad, N / len)
len \ N 10k 100k 1M 10M 100M
1 0.088 / 0.056 0.422 / 0.102 3.88 / 0.83 39.5 / 9.1 393 / 97
10 0.061 / 0.056 0.128 / 0.102 1.10 / 0.83 11.5 / 8.9 121 / 97
100 0.058 / 0.058 0.105 / 0.102 0.85 / 0.83 9.2 / 8.9 96 / 96
1000 0.058 / 0.059 0.106 / 0.105 0.84 / 0.83 9.2 / 9.0 95 / 95
10000 0.078 / 0.083 0.123 / 0.124 0.85 / 0.86 8.9 / 9.2 97 / 95
# rpad('x', N, pad)
len \ N 10k 100k 1M 10M 100M
1 0.106 / 0.057 0.548 / 0.061 4.80 / 0.11 53.7 / 5.7 560 / 97
10 0.105 / 0.057 0.543 / 0.061 4.83 / 0.12 52.5 / 5.6 564 / 97
100 0.107 / 0.058 0.560 / 0.062 4.89 / 0.12 53.6 / 5.5 564 / 96
1000 0.109 / 0.064 0.563 / 0.069 4.89 / 0.13 53.5 / 5.8 560 / 96
10000 0.129 / 0.107 0.576 / 0.129 5.02 / 0.18 53.2 / 5.7 566 / 97
Tests are the lpad/rpad set from v3 plus some additional ones to go
past the 16K doubling limit.
On this machine, block sizes from 16 kB up to 4 MB performed the same
within noise, while 16 MB was consistently slower for large results,
much like v4's pure doubling. So I don't see a reason to go bigger
than 16 kB.
The doubling approach for repeat() was proposed by Chenhui Mo in [1].
Jeevan Chalke, Heikki Linnakangas, David Rowley, and Jan Nidzwetzki
also discussed and benchmarked capped/fixed-size variants there, and
that work shaped the fixed-size block used here.
This series overlaps with the repeat() work linked above, so the two are not
independent. The v2 series there, currently marked Ready for Committer,
contains the empty-result and single-byte memset() changes but leaves
the doubling change out.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
Attachments: download all (3 files, .tar.gz)
v5-0001-Factor-the-padding-loop-out-of-lpad-and-rpad.patch (application/x-patch, 3.0K) download
From 0c152b84af6ebed24bc65fc62682bed80db2c9a7 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 23 Sep 2026 11:47:02 -0500
Subject: [PATCH v5 1/3] Factor the padding loop out of lpad() and rpad().
lpad() and rpad() each carry an identical copy of the loop that
writes the padding characters. This commit moves it into a helper
function that both call. No functional change.
This is preparatory work for a follow-up commit that will teach the
helper to write long padding with a few large memcpy() calls instead
of one per character.
Discussion: https://postgr.es/m/CAH7T-apj+pFg9bRkXVGfGejS2Uu4WzdMd7KoLKTW11mdsrt1Ew@mail.gmail.com
---
src/backend/utils/adt/oracle_compat.c | 60 ++++++++++++---------------
1 file changed, 27 insertions(+), 33 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 7422a454397..e5238e44813 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -143,6 +143,31 @@ casefold(PG_FUNCTION_ARGS)
}
+/*
+ * Append m characters of the padding string pad (padlen bytes) to dst,
+ * cycling through pad as needed, and return a pointer past the last byte
+ * written.
+ */
+static char *
+append_padding(char *dst, const char *pad, int padlen, int m)
+{
+ const char *p = pad;
+ const char *pend = pad + padlen;
+
+ while (m--)
+ {
+ int mlen = pg_mblen_range(p, pend);
+
+ memcpy(dst, p, mlen);
+ dst += mlen;
+ p += mlen;
+ if (p == pend) /* wrap around at end of pad */
+ p = pad;
+ }
+
+ return dst;
+}
+
/********************************************************************
*
* lpad
@@ -167,10 +192,7 @@ lpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -209,20 +231,7 @@ lpad(PG_FUNCTION_ARGS)
m = len - s1len;
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
- ptr_ret = VARDATA(ret);
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(VARDATA(ret), VARDATA_ANY(string2), s2len, m);
ptr1 = VARDATA_ANY(string1);
@@ -265,10 +274,7 @@ rpad(PG_FUNCTION_ARGS)
text *string2 = PG_GETARG_TEXT_PP(2);
text *ret;
char *ptr1,
- *ptr2,
- *ptr2start,
*ptr_ret;
- const char *ptr2end;
int m,
s1len,
s2len;
@@ -320,19 +326,7 @@ rpad(PG_FUNCTION_ARGS)
ptr1 += mlen;
}
- ptr2 = ptr2start = VARDATA_ANY(string2);
- ptr2end = ptr2 + s2len;
-
- while (m--)
- {
- int mlen = pg_mblen_range(ptr2, ptr2end);
-
- memcpy(ptr_ret, ptr2, mlen);
- ptr_ret += mlen;
- ptr2 += mlen;
- if (ptr2 == ptr2end) /* wrap around at end of s2 */
- ptr2 = ptr2start;
- }
+ ptr_ret = append_padding(ptr_ret, VARDATA_ANY(string2), s2len, m);
SET_VARSIZE(ret, ptr_ret - (char *) ret);
--
2.55.0
v5-0003-Use-repeat_bytes-in-repeat.patch (application/x-patch, 3.2K) download
From 666fdb5b536342bcdf1edcc37e69eed39e7b13d1 Mon Sep 17 00:00:00 2001
From: Sehrope Sarkuni <sehrope@jackdb.com>
Date: Thu, 24 Sep 2026 11:56:07 +0000
Subject: [PATCH v5 3/3] Use repeat_bytes() in repeat()
repeat() copied the string once per repetition. Use repeat_bytes()
instead, which writes a large result with a few doubling copies and
then block copies, and checks for interrupts between the latter as
the old loop did between copies.
Discussion: https://postgr.es/m/CAH7T-apj+pFg9bRkXVGfGejS2Uu4WzdMd7KoLKTW11mdsrt1Ew@mail.gmail.com
Discussion: https://postgr.es/m/tencent_C5BBECF985A270FBC49463EDAF722CD5E005@qq.com
---
src/backend/utils/adt/oracle_compat.c | 12 +-----------
src/test/regress/expected/strings.out | 19 +++++++++++++++++++
src/test/regress/sql/strings.sql | 5 +++++
3 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index 6fdbd595a9f..fb584aa8b06 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -1236,9 +1236,6 @@ repeat(PG_FUNCTION_ARGS)
text *result;
int slen,
tlen;
- int i;
- char *cp,
- *sp;
if (count < 0)
count = 0;
@@ -1255,14 +1252,7 @@ repeat(PG_FUNCTION_ARGS)
result = (text *) palloc(tlen);
SET_VARSIZE(result, tlen);
- cp = VARDATA(result);
- sp = VARDATA_ANY(string);
- for (i = 0; i < count; i++)
- {
- memcpy(cp, sp, slen);
- cp += slen;
- CHECK_FOR_INTERRUPTS();
- }
+ repeat_bytes(VARDATA(result), VARDATA_ANY(string), slen, count);
PG_RETURN_TEXT_P(result);
}
diff --git a/src/test/regress/expected/strings.out b/src/test/regress/expected/strings.out
index b588398d7fe..80669ab3406 100644
--- a/src/test/regress/expected/strings.out
+++ b/src/test/regress/expected/strings.out
@@ -3474,6 +3474,25 @@ SELECT rpad('', 50000, 'abc') = (SELECT string_agg('abc', '') FROM generate_seri
t
(1 row)
+-- repeat: one, several and five copies, and empty results
+SELECT repeat('abc', 1), repeat('abc', 2), repeat('ab', 5);
+ repeat | repeat | repeat
+--------+--------+------------
+ abc | abcabc | ababababab
+(1 row)
+
+SELECT repeat('abc', 0), repeat('', 5), repeat('abc', -1);
+ repeat | repeat | repeat
+--------+--------+--------
+ | |
+(1 row)
+
+SELECT repeat('abc', 10000) = (SELECT string_agg('abc', '') FROM generate_series(1, 10000));
+ ?column?
+----------
+ t
+(1 row)
+
SELECT ltrim('zzzytrim', 'xyz');
ltrim
-------
diff --git a/src/test/regress/sql/strings.sql b/src/test/regress/sql/strings.sql
index 5db0df1c6ae..1ee524543a6 100644
--- a/src/test/regress/sql/strings.sql
+++ b/src/test/regress/sql/strings.sql
@@ -1174,6 +1174,11 @@ SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
-- padding longer than the 16 kB block that repeat_bytes() copies
SELECT rpad('', 50000, 'abc') = (SELECT string_agg('abc', '') FROM generate_series(1, 16666)) || 'ab';
+-- repeat: one, several and five copies, and empty results
+SELECT repeat('abc', 1), repeat('abc', 2), repeat('ab', 5);
+SELECT repeat('abc', 0), repeat('', 5), repeat('abc', -1);
+SELECT repeat('abc', 10000) = (SELECT string_agg('abc', '') FROM generate_series(1, 10000));
+
SELECT ltrim('zzzytrim', 'xyz');
SELECT translate('', '14', 'ax');
--
2.55.0
v5-0002-Copy-whole-repetitions-of-the-pad-string-at-once-.patch (application/x-patch, 9.7K) download
From 6dcad3ba7bd74e8799a4cf8a129f1918d49aa20d Mon Sep 17 00:00:00 2001
From: Sehrope Sarkuni <sehrope@jackdb.com>
Date: Thu, 24 Sep 2026 11:55:30 +0000
Subject: [PATCH v5 2/3] Copy whole repetitions of the pad string at once in
lpad() and rpad()
append_padding() called pg_mblen_range() and memcpy() once per
character. Walk the pad string once to count its characters, write
the whole repetitions with a new repeat_bytes() helper, and copy the
partial final repetition as bytes. The pad string is validated as far
as before, which is all of it when a full repetition is copied and
otherwise only the characters copied.
repeat_bytes() copies the string once, doubles the copied region
until it is at least 16 kB, and then copies that region repeatedly,
checking for interrupts between copies. Doubling needs only a few
memcpy() calls for short strings, and the fixed block stays in cache
for large results, which pure doubling did not. This also makes long
lpad() and rpad() calls cancellable, which they were not before.
Co-authored-by: Nathan Bossart <nathan@postgresql.org>
Discussion: https://postgr.es/m/CAH7T-apj+pFg9bRkXVGfGejS2Uu4WzdMd7KoLKTW11mdsrt1Ew@mail.gmail.com
---
src/backend/utils/adt/oracle_compat.c | 93 +++++++++++++++++++++++---
src/test/regress/expected/encoding.out | 37 ++++++++++
src/test/regress/expected/strings.out | 33 +++++++++
src/test/regress/sql/encoding.sql | 10 +++
src/test/regress/sql/strings.sql | 9 +++
5 files changed, 174 insertions(+), 8 deletions(-)
diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c
index e5238e44813..6fdbd595a9f 100644
--- a/src/backend/utils/adt/oracle_compat.c
+++ b/src/backend/utils/adt/oracle_compat.c
@@ -143,29 +143,106 @@ casefold(PG_FUNCTION_ARGS)
}
+/*
+ * Write count copies of the srclen-byte string src to dst and return a
+ * pointer past the last byte written.
+ *
+ * The first copy is written with memcpy(), then the region written so far
+ * is copied onto its own end, doubling it each time, until it is at least
+ * REPEAT_BYTES_BLOCK long. From there on that region is copied repeatedly.
+ * The doubling keeps the number of memcpy() calls small for short strings,
+ * and the fixed block, which is a whole number of copies and stays in
+ * cache, keeps the source reads cheap for large outputs. The caller must
+ * have checked that count * srclen bytes fit in dst.
+ */
+#define REPEAT_BYTES_BLOCK 16384
+
+static char *
+repeat_bytes(char *dst, const char *src, int srclen, int count)
+{
+ Size total = (Size) srclen * count;
+ Size written;
+ Size block;
+
+ if (total == 0)
+ return dst;
+ Assert(srclen > 0 && count > 0);
+
+ memcpy(dst, src, srclen);
+ written = srclen;
+ while (written < total && written < REPEAT_BYTES_BLOCK)
+ {
+ Size n = Min(written, total - written);
+
+ memcpy(dst + written, dst, n);
+ written += n;
+ }
+
+ block = written;
+ while (written < total)
+ {
+ Size n = Min(block, total - written);
+
+ memcpy(dst + written, dst, n);
+ written += n;
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ return dst + total;
+}
+
/*
* Append m characters of the padding string pad (padlen bytes) to dst,
* cycling through pad as needed, and return a pointer past the last byte
* written.
+ *
+ * The pad string is validated with pg_mblen_range() only as far as it is
+ * used, so an incomplete multibyte character at its end is an error only
+ * if the padding reaches it.
*/
static char *
append_padding(char *dst, const char *pad, int padlen, int m)
{
const char *p = pad;
const char *pend = pad + padlen;
+ int nchars = 0;
+ int nrep;
+ int tail;
+
+ Assert(padlen > 0 || m <= 0);
+
+ if (m <= 0 || padlen <= 0)
+ return dst;
- while (m--)
+ /* count the characters of one repetition, stopping at m */
+ while (p < pend && nchars < m)
{
- int mlen = pg_mblen_range(p, pend);
+ p += pg_mblen_range(p, pend);
+ nchars++;
+ }
- memcpy(dst, p, mlen);
- dst += mlen;
- p += mlen;
- if (p == pend) /* wrap around at end of pad */
- p = pad;
+ /* whole repetitions only if the pad string was counted to its end */
+ nrep = (p == pend) ? m / nchars : 0;
+ if (nrep == 0)
+ {
+ /* the loop above stopped after exactly m characters */
+ memcpy(dst, pad, p - pad);
+ return dst + (p - pad);
}
- return dst;
+ dst = repeat_bytes(dst, pad, padlen, nrep);
+
+ /*
+ * The remaining m - nrep * nchars characters are a prefix of pad that the
+ * loop above already checked, so measure them with the unbounded variant
+ * and copy them as bytes.
+ */
+ p = pad;
+ for (tail = m - nrep * nchars; tail > 0; tail--)
+ p += pg_mblen_unbounded(p);
+ memcpy(dst, pad, p - pad);
+
+ return dst + (p - pad);
}
/********************************************************************
diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out
index 0bb72a1df6f..9fc871215b0 100644
--- a/src/test/regress/expected/encoding.out
+++ b/src/test/regress/expected/encoding.out
@@ -60,6 +60,43 @@ SELECT reverse(good) FROM regress_encoding;
éfac
(1 row)
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+ lpad | rpad
+---------+---------
+ ééécafé | caféééé
+(1 row)
+
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+ lpad | rpad
+--------------+--------------
+ éabéabéacafé | cafééabéabéa
+(1 row)
+
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+ lpad | rpad
+-------+-------
+ écafé | caféé
+(1 row)
+
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ERROR: invalid byte sequence for encoding "UTF8": 0xc3
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+-------+--------
+ acafé | caféab
+(1 row)
+
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
+ lpad | rpad
+------+------
+ café | café
+(1 row)
+
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
ERROR: invalid byte sequence for encoding "UTF8": 0xc3
diff --git a/src/test/regress/expected/strings.out b/src/test/regress/expected/strings.out
index fa29abfd829..b588398d7fe 100644
--- a/src/test/regress/expected/strings.out
+++ b/src/test/regress/expected/strings.out
@@ -3441,6 +3441,39 @@ SELECT rpad('hi', 5, '');
hi
(1 row)
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+ lpad | rpad
+----------+----------
+ abcabchi | hiabcabc
+(1 row)
+
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+ lpad | rpad
+-----------+-----------
+ abcabcahi | hiabcabca
+(1 row)
+
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+ lpad | rpad
+--------------+--------------
+ abababababhi | hiababababab
+(1 row)
+
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+ lpad | rpad
+------+------
+ ahi | hia
+(1 row)
+
+-- padding longer than the 16 kB block that repeat_bytes() copies
+SELECT rpad('', 50000, 'abc') = (SELECT string_agg('abc', '') FROM generate_series(1, 16666)) || 'ab';
+ ?column?
+----------
+ t
+(1 row)
+
SELECT ltrim('zzzytrim', 'xyz');
ltrim
-------
diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql
index 26caa93a5d5..3ea6e54e52d 100644
--- a/src/test/regress/sql/encoding.sql
+++ b/src/test/regress/sql/encoding.sql
@@ -37,6 +37,16 @@ SELECT substring(good, 3, 1) FROM regress_encoding;
SELECT substring(good, 4, 1) FROM regress_encoding;
SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding;
SELECT reverse(good) FROM regress_encoding;
+-- multibyte pad strings: whole repetitions, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding;
+SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding;
+SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding;
+-- a lone lead byte in the pad string is an error if the padding reaches it
+SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding;
+SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding;
-- invalid short mb character = error
SELECT length(truncated) FROM regress_encoding;
diff --git a/src/test/regress/sql/strings.sql b/src/test/regress/sql/strings.sql
index 7d9c7275a02..5db0df1c6ae 100644
--- a/src/test/regress/sql/strings.sql
+++ b/src/test/regress/sql/strings.sql
@@ -1165,6 +1165,15 @@ SELECT rpad('hi', -5, 'xy');
SELECT rpad('hello', 2);
SELECT rpad('hi', 5, '');
+-- whole repetitions of the pad string, a partial final repetition, and
+-- fewer than one repetition
+SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc');
+SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc');
+SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab');
+SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc');
+-- padding longer than the 16 kB block that repeat_bytes() copies
+SELECT rpad('', 50000, 'abc') = (SELECT string_agg('abc', '') FROM generate_series(1, 16666)) || 'ab';
+
SELECT ltrim('zzzytrim', 'xyz');
SELECT translate('', '14', 'ax');
--
2.55.0
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-09-25 01:24 Manu <manuelreyesbravo@gmail.com>
parent: Sehrope Sarkuni <sehrope@jackdb.com>
0 siblings, 1 reply; 12+ messages in thread
From: Manu @ 2026-09-25 01:24 UTC (permalink / raw)
To: Sehrope Sarkuni <sehrope@jackdb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Nathan Bossart <nathandbossart@gmail.com>; Jan Nidzwetzki <jan@planetscale.com>; ChenhuiMo <chenhuimo.mch@qq.com>; pgsql-hackers
Hi,
Sehrope Sarkuni <sehrope@jackdb.com> wrote:
> On this machine, block sizes from 16 kB up to 4 MB performed the same
> within noise, while 16 MB was consistently slower for large results,
> much like v4's pure doubling. So I don't see a reason to go bigger
> than 16 kB.
I ran v5 on a different x86 machine: an AMD Ryzen AI 9 HX 370 (Zen 5),
glibc 2.43, where x86_non_temporal_threshold is 6 MiB. Release builds
of master 2c10c2ce4d7, with your method: pgbench -c 1, rounds
alternating between builds, median of 5. This CPU mixes Zen 5 and
Zen 5c cores, so the server was pinned to one Zen 5 core. I also
included Chenhui's v2 from the repeat() thread (Ready for Committer),
since 0003 replaces the same loop.
1. Block size
The repeat() thread found the best block to be the first one at or
above glibc's non-temporal threshold, on three Intel machines. To
test that here, I built v5 with REPEAT_BYTES_BLOCK = 8 MB, above the
6 MiB threshold, and ran that build a second time with non-temporal
stores disabled through GLIBC_TUNABLES.
At 100 MB the three stay within the round-to-round spread (about 10%)
for repeat() and rpad() with 1-, 10- and 100-byte sources. For
example, repeat() with a 10-byte source took 46.8 ms with 16 kB,
50.4 ms with 8 MB, and 46.7 ms with 8 MB and no non-temporal stores.
So the effect seen on the Intel machines does not show up on this
Zen 5, and 16 kB costs nothing here.
2. repeat() against the Ready for Committer patch
- 1-byte source, 100 MB: master 173.1 ms, v5 47.2 ms, RfC 47.7 ms
- 1-byte source, 16 MB: master 27.3 ms, v5 7.0 ms, RfC 6.7 ms
- 10-byte source, 100 MB: master 58.6 ms, v5 46.8 ms, RfC 55.3 ms
- 100-byte source, 100 MB: master 46.8 ms, v5 44.9 ms
So on this machine dropping the memset() path costs nothing for a
one-byte source, and v5 also speeds up the 10-byte case, which the RfC
patch does not touch. Jan's M5 Max numbers upthread had chunked
memset() well behind a single one. I have no ARM machine to check
whether that carries over to repeat_bytes().
3. rpad(), 100 MB
- 1-byte pad: master 330.6 ms, v5 45.5 ms
- 10-byte pad: master 342.5 ms, v5 44.6 ms
- 100-byte pad: master 325.9 ms, v5 43.2 ms
4. Results are unchanged
I compared lpad(), rpad() and repeat() output between master and v5
in UTF8, EUC_JP and SQL_ASCII databases: 6075 calls, with lengths
around the 16 kB and 8 MB boundaries, multibyte and mixed pads with
partial tails, and sizes near the 1 GB limit. Lengths, md5 of every
result and error messages are identical, including the same 16
"requested length too large" errors. Invalid pad strings are not in
it, since the encoding.sql tests cover those.
5. Cancellation
On master, lpad() and rpad() have no CHECK_FOR_INTERRUPTS() in the
padding loop. With statement_timeout = 20ms,
rpad('x', 268000000, 'ab') runs for about 900 ms, the whole call,
before the cancel takes effect. With v5 it is cancelled after about
24 ms, as repeat() already is on master (21 ms), because the padding
now goes through repeat_bytes().
Scripts, raw timings and outputs are in the attachment.
Regards,
Manu
Attachments:
nocfbot-pad-v5-review.txt (text/plain, 31.9K) download
Review of v5 (lpad/rpad/repeat): timings, differential check, cancellation
master 2c10c2ce4d7; release builds (default -O2, no asserts)
===== machine =====
Model name: AMD Ryzen AI 9 HX 370 w/ Radeon 890M
L1d cache: 576 KiB (12 instances)
L2 cache: 12 MiB (12 instances)
L3 cache: 24 MiB (2 instances)
cores 0-3 Zen 5, 4-11 Zen 5c; server pinned to CPU 0, pgbench/psql to CPU 2
ldd (GNU libc) 2.43
glibc.cpu.x86_data_cache_size: 0xc000 (min: 0x0, max: 0xffffffffffffffff)
glibc.cpu.x86_memset_non_temporal_threshold: 0x600000 (min: 0x4040, max: 0xffffffffffffffff)
glibc.cpu.x86_non_temporal_threshold: 0x600000 (min: 0x4040, max: 0xfffffffffffffff)
glibc.cpu.x86_rep_movsb_threshold: 0x600000 (min: 0x200, max: 0xffffffffffffffff)
glibc.cpu.x86_shared_cache_size: 0x800000 (min: 0x0, max: 0xffffffffffffffff)
"===== build.sh ====="
#!/bin/bash
# Release builds (no asserts, default -O2) from the same master commit:
# master control
# v5 Sehrope's v5 (0001-0003), REPEAT_BYTES_BLOCK = 16 kB
# v5b8m v5 with REPEAT_BYTES_BLOCK = 8 MB, the only change
# rfc Chenhui's repeat() v2 (Ready for Committer): early return + memset()
set -eu
BASE=${BASE:-2c10c2ce4d7}
SRC=$HOME/Proyectos/postgresql
W=$HOME/pgpad
A=$(cd "$(dirname "$0")" && pwd)
mkdir -p $W
tree() { # name patch...
local name=$1; shift
local t=$W/src-$name
[ -d $t ] && return
git -C $SRC worktree add -q --detach $t $BASE
for p in "$@"; do git -C $t am -q "$p"; done
}
tree master
tree v5 $A/sehrope-v5/v5-000{1,2,3}-*.patch
tree v5b8m $A/sehrope-v5/v5-000{1,2,3}-*.patch
sed -i 's/^#define REPEAT_BYTES_BLOCK\t16384$/#define REPEAT_BYTES_BLOCK\t(8 * 1024 * 1024)/' \
$W/src-v5b8m/src/backend/utils/adt/oracle_compat.c $W/src-v5b8m/src/backend/utils/adt/varlena.c 2>/dev/null || true
grep -rn 'define REPEAT_BYTES_BLOCK' $W/src-v5b8m/src/backend/utils/adt/
tree rfc $A/chenhui-v2/v2-000{1,2}-*.patch
build() {
local name=$1
mkdir -p $W/b-$name && cd $W/b-$name
$W/src-$name/configure --prefix=$W/i-$name --quiet > configure.log 2>&1
make -j"$(nproc)" -s > build.log 2>&1
make -s install > install.log 2>&1
echo "$name: $(git -C $W/src-$name log -1 --format='%h %s' | cut -c1-60) warnings=$(grep -c 'warning:' build.log)"
}
build master & build v5 & build v5b8m & build rfc & wait
echo BUILD-DONE
"===== bench.sh ====="
#!/bin/bash
# repeat() and rpad() timings on master and the patched builds, following
# Sehrope's method: pgbench -c 1, one statement per transaction, rounds
# alternating between builds, median of ROUNDS.
#
# Configs (build[:env]):
# master v5 v5b8m rfc
# v5b8m-nont = v5b8m with glibc's non-temporal stores disabled
# (GLIBC_TUNABLES), to see whether a gain comes from them
#
# This CPU mixes Zen 5 and Zen 5c cores, so the server runs pinned to
# SRVCPU and pgbench to CLICPU.
# bench.sh [ROUNDS]
set -u
ROUNDS=${1:-5}
A=$(cd "$(dirname "$0")" && pwd)
W=$HOME/pgpad
SRVCPU=${SRVCPU:-0}; CLICPU=${CLICPU:-2}
CONFIGS=${CONFIGS:-"master v5 v5b8m v5b8m-nont rfc"}
# glibc 2.43 silently keeps the default when the value equals the listed max
# of the memset tunable (0xffffffffffffffff), so both use 0xfffffffffffffff;
# check with: GLIBC_TUNABLES=$NONT /lib64/ld-linux-x86-64.so.2 --list-tunables
# (the first run of this script had the memset one ignored; v5 does not use
# memset(), so that run's v5b8m-nont numbers are unaffected)
NONT='glibc.cpu.x86_non_temporal_threshold=0xfffffffffffffff:glibc.cpu.x86_memset_non_temporal_threshold=0xfffffffffffffff'
PAD=abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij
OUT=$A/bench.raw; : > $OUT
port=55480
declare -A PORT DIR
for c in $CONFIGS; do
b=${c%-nont}
D=$(mktemp -d /tmp/claude-1000/pad.XXXX); DIR[$c]=$D; PORT[$c]=$port
$W/i-$b/bin/initdb -D $D -A trust --no-sync -U postgres >/dev/null
printf "port = $port\nunix_socket_directories = '/tmp'\nmax_connections = 5\n" >> $D/postgresql.conf
if [ "$c" != "$b" ]; then
GLIBC_TUNABLES=$NONT taskset -c $SRVCPU $W/i-$b/bin/pg_ctl -D $D -l $D/log -w start >/dev/null
else
taskset -c $SRVCPU $W/i-$b/bin/pg_ctl -D $D -l $D/log -w start >/dev/null
fi
port=$((port + 1))
done
# case lines: name|sql|transactions per round
cases() {
for len in 1 10 100; do
s=${PAD:0:$len}
for n in 1000000 16000000 100000000; do
t=$(( n >= 100000000 ? 5 : (n >= 16000000 ? 20 : 200) ))
echo "repeat len=$len N=$n|SELECT octet_length(repeat('$s', $((n / len))))|$t"
echo "rpad len=$len N=$n|SELECT octet_length(rpad('x', $n, '$s'))|$t"
done
done
}
for r in $(seq $ROUNDS); do
# rotate the order of the configs each round
set -- $CONFIGS; for _ in $(seq $(( (r - 1) % $# ))); do set -- "${@:2}" "$1"; done
order="$*"
cases | while IFS='|' read -r name sql t; do
for c in $order; do
b=${c%-nont}
f=$(mktemp /tmp/claude-1000/q.XXXX.sql); echo "$sql;" > $f
lat=$(taskset -c $CLICPU $W/i-$b/bin/pgbench -n -h /tmp -p ${PORT[$c]} -U postgres -c 1 -t $t -f $f postgres 2>/dev/null \
| awk '/latency average/ {print $4}')
rm -f $f
echo "$r|$c|$name|$lat" >> $OUT
done
done
echo "round $r done" >&2
done
for c in $CONFIGS; do
b=${c%-nont}
$W/i-$b/bin/pg_ctl -D ${DIR[$c]} -m fast -w stop >/dev/null; rm -rf ${DIR[$c]}
done
# median per (config, case), one line per case: master first, then the rest
python3 - "$OUT" "$CONFIGS" <<'EOF' | tee $A/bench.txt
import sys, statistics, collections
rows = [l.rstrip("\n").split("|") for l in open(sys.argv[1]) if l.strip()]
configs = sys.argv[2].split()
med = collections.defaultdict(list)
order = []
for r, c, name, lat in rows:
if lat:
med[(c, name)].append(float(lat))
if name not in order:
order.append(name)
print("median latency in ms, " + ", ".join(configs))
for name in order:
vals = [statistics.median(med[(c, name)]) if med[(c, name)] else float("nan") for c in configs]
print(f"{name}: " + ", ".join(f"{v:.3f}" for v in vals))
EOF
===== bench.txt (./bench.sh 5) =====
median latency in ms, master, v5, v5b8m, v5b8m-nont, rfc
repeat len=1 N=1000000: 1.487, 0.211, 0.228, 0.216, 0.219
rpad len=1 N=1000000: 2.949, 0.043, 0.042, 0.045, 2.924
repeat len=1 N=16000000: 27.279, 6.995, 7.623, 7.136, 6.695
rpad len=1 N=16000000: 49.933, 4.108, 4.056, 3.730, 50.064
repeat len=1 N=100000000: 173.132, 47.155, 46.926, 47.027, 47.727
rpad len=1 N=100000000: 330.626, 45.497, 47.149, 48.070, 330.721
repeat len=10 N=1000000: 0.327, 0.220, 0.210, 0.246, 0.327
rpad len=10 N=1000000: 2.999, 0.038, 0.043, 0.043, 2.939
repeat len=10 N=16000000: 8.480, 6.071, 5.930, 6.239, 8.619
rpad len=10 N=16000000: 50.406, 3.777, 4.464, 3.771, 51.329
repeat len=10 N=100000000: 58.642, 46.783, 50.412, 46.664, 55.334
rpad len=10 N=100000000: 342.535, 44.577, 46.989, 46.646, 328.291
repeat len=100 N=1000000: 0.239, 0.229, 0.226, 0.223, 0.262
rpad len=100 N=1000000: 2.929, 0.048, 0.052, 0.060, 2.983
repeat len=100 N=16000000: 6.348, 6.251, 6.729, 6.429, 6.750
rpad len=100 N=16000000: 51.905, 3.849, 4.138, 4.208, 50.296
repeat len=100 N=100000000: 46.811, 44.867, 47.551, 48.362, 46.295
rpad len=100 N=100000000: 325.854, 43.249, 48.334, 50.560, 335.755
===== bench.raw (round|config|case|latency ms) =====
1|master|repeat len=1 N=1000000|1.573
1|v5|repeat len=1 N=1000000|0.211
1|v5b8m|repeat len=1 N=1000000|0.228
1|v5b8m-nont|repeat len=1 N=1000000|0.276
1|rfc|repeat len=1 N=1000000|0.233
1|master|rpad len=1 N=1000000|3.044
1|v5|rpad len=1 N=1000000|0.051
1|v5b8m|rpad len=1 N=1000000|0.044
1|v5b8m-nont|rpad len=1 N=1000000|0.051
1|rfc|rpad len=1 N=1000000|2.921
1|master|repeat len=1 N=16000000|27.815
1|v5|repeat len=1 N=16000000|6.995
1|v5b8m|repeat len=1 N=16000000|7.710
1|v5b8m-nont|repeat len=1 N=16000000|7.377
1|rfc|repeat len=1 N=16000000|7.540
1|master|rpad len=1 N=16000000|52.476
1|v5|rpad len=1 N=16000000|3.600
1|v5b8m|rpad len=1 N=16000000|4.056
1|v5b8m-nont|rpad len=1 N=16000000|3.650
1|rfc|rpad len=1 N=16000000|50.327
1|master|repeat len=1 N=100000000|168.610
1|v5|repeat len=1 N=100000000|46.517
1|v5b8m|repeat len=1 N=100000000|46.926
1|v5b8m-nont|repeat len=1 N=100000000|46.215
1|rfc|repeat len=1 N=100000000|47.727
1|master|rpad len=1 N=100000000|330.626
1|v5|rpad len=1 N=100000000|43.893
1|v5b8m|rpad len=1 N=100000000|45.238
1|v5b8m-nont|rpad len=1 N=100000000|46.845
1|rfc|rpad len=1 N=100000000|327.350
1|master|repeat len=10 N=1000000|0.317
1|v5|repeat len=10 N=1000000|0.220
1|v5b8m|repeat len=10 N=1000000|0.210
1|v5b8m-nont|repeat len=10 N=1000000|0.261
1|rfc|repeat len=10 N=1000000|0.334
1|master|rpad len=10 N=1000000|3.134
1|v5|rpad len=10 N=1000000|0.038
1|v5b8m|rpad len=10 N=1000000|0.042
1|v5b8m-nont|rpad len=10 N=1000000|0.050
1|rfc|rpad len=10 N=1000000|2.939
1|master|repeat len=10 N=16000000|8.018
1|v5|repeat len=10 N=16000000|5.506
1|v5b8m|repeat len=10 N=16000000|5.930
1|v5b8m-nont|repeat len=10 N=16000000|5.743
1|rfc|repeat len=10 N=16000000|8.459
1|master|rpad len=10 N=16000000|52.242
1|v5|rpad len=10 N=16000000|3.798
1|v5b8m|rpad len=10 N=16000000|5.439
1|v5b8m-nont|rpad len=10 N=16000000|5.600
1|rfc|rpad len=10 N=16000000|52.484
1|master|repeat len=10 N=100000000|58.642
1|v5|repeat len=10 N=100000000|46.918
1|v5b8m|repeat len=10 N=100000000|52.342
1|v5b8m-nont|repeat len=10 N=100000000|50.504
1|rfc|repeat len=10 N=100000000|58.684
1|master|rpad len=10 N=100000000|328.584
1|v5|rpad len=10 N=100000000|46.283
1|v5b8m|rpad len=10 N=100000000|46.989
1|v5b8m-nont|rpad len=10 N=100000000|47.474
1|rfc|rpad len=10 N=100000000|332.579
1|master|repeat len=100 N=1000000|0.237
1|v5|repeat len=100 N=1000000|0.206
1|v5b8m|repeat len=100 N=1000000|0.213
1|v5b8m-nont|repeat len=100 N=1000000|0.206
1|rfc|repeat len=100 N=1000000|0.224
1|master|rpad len=100 N=1000000|2.868
1|v5|rpad len=100 N=1000000|0.048
1|v5b8m|rpad len=100 N=1000000|0.062
1|v5b8m-nont|rpad len=100 N=1000000|0.063
1|rfc|rpad len=100 N=1000000|3.028
1|master|repeat len=100 N=16000000|7.206
1|v5|repeat len=100 N=16000000|6.326
1|v5b8m|repeat len=100 N=16000000|6.675
1|v5b8m-nont|repeat len=100 N=16000000|6.117
1|rfc|repeat len=100 N=16000000|6.750
1|master|rpad len=100 N=16000000|49.635
1|v5|rpad len=100 N=16000000|4.055
1|v5b8m|rpad len=100 N=16000000|4.138
1|v5b8m-nont|rpad len=100 N=16000000|4.208
1|rfc|rpad len=100 N=16000000|50.296
1|master|repeat len=100 N=100000000|46.811
1|v5|repeat len=100 N=100000000|38.952
1|v5b8m|repeat len=100 N=100000000|43.863
1|v5b8m-nont|repeat len=100 N=100000000|47.189
1|rfc|repeat len=100 N=100000000|45.777
1|master|rpad len=100 N=100000000|322.829
1|v5|rpad len=100 N=100000000|42.335
1|v5b8m|rpad len=100 N=100000000|45.797
1|v5b8m-nont|rpad len=100 N=100000000|47.126
1|rfc|rpad len=100 N=100000000|336.829
2|v5|repeat len=1 N=1000000|0.210
2|v5b8m|repeat len=1 N=1000000|0.245
2|v5b8m-nont|repeat len=1 N=1000000|0.208
2|rfc|repeat len=1 N=1000000|0.202
2|master|repeat len=1 N=1000000|1.487
2|v5|rpad len=1 N=1000000|0.035
2|v5b8m|rpad len=1 N=1000000|0.041
2|v5b8m-nont|rpad len=1 N=1000000|0.042
2|rfc|rpad len=1 N=1000000|2.886
2|master|rpad len=1 N=1000000|2.930
2|v5|repeat len=1 N=16000000|7.046
2|v5b8m|repeat len=1 N=16000000|6.534
2|v5b8m-nont|repeat len=1 N=16000000|6.739
2|rfc|repeat len=1 N=16000000|6.244
2|master|repeat len=1 N=16000000|25.690
2|v5|rpad len=1 N=16000000|4.209
2|v5b8m|rpad len=1 N=16000000|4.004
2|v5b8m-nont|rpad len=1 N=16000000|3.687
2|rfc|rpad len=1 N=16000000|48.573
2|master|rpad len=1 N=16000000|49.657
2|v5|repeat len=1 N=100000000|47.155
2|v5b8m|repeat len=1 N=100000000|55.758
2|v5b8m-nont|repeat len=1 N=100000000|47.238
2|rfc|repeat len=1 N=100000000|43.216
2|master|repeat len=1 N=100000000|164.346
2|v5|rpad len=1 N=100000000|43.217
2|v5b8m|rpad len=1 N=100000000|47.149
2|v5b8m-nont|rpad len=1 N=100000000|56.707
2|rfc|rpad len=1 N=100000000|340.892
2|master|rpad len=1 N=100000000|325.681
2|v5|repeat len=10 N=1000000|0.210
2|v5b8m|repeat len=10 N=1000000|0.206
2|v5b8m-nont|repeat len=10 N=1000000|0.246
2|rfc|repeat len=10 N=1000000|0.514
2|master|repeat len=10 N=1000000|0.473
2|v5|rpad len=10 N=1000000|0.036
2|v5b8m|rpad len=10 N=1000000|0.045
2|v5b8m-nont|rpad len=10 N=1000000|0.043
2|rfc|rpad len=10 N=1000000|2.893
2|master|rpad len=10 N=1000000|2.857
2|v5|repeat len=10 N=16000000|5.536
2|v5b8m|repeat len=10 N=16000000|5.716
2|v5b8m-nont|repeat len=10 N=16000000|5.801
2|rfc|repeat len=10 N=16000000|7.902
2|master|repeat len=10 N=16000000|7.745
2|v5|rpad len=10 N=16000000|3.293
2|v5b8m|rpad len=10 N=16000000|3.757
2|v5b8m-nont|rpad len=10 N=16000000|3.173
2|rfc|rpad len=10 N=16000000|48.776
2|master|rpad len=10 N=16000000|48.802
2|v5|repeat len=10 N=100000000|40.933
2|v5b8m|repeat len=10 N=100000000|42.228
2|v5b8m-nont|repeat len=10 N=100000000|46.664
2|rfc|repeat len=10 N=100000000|55.334
2|master|repeat len=10 N=100000000|53.506
2|v5|rpad len=10 N=100000000|43.690
2|v5b8m|rpad len=10 N=100000000|50.387
2|v5b8m-nont|rpad len=10 N=100000000|50.047
2|rfc|rpad len=10 N=100000000|328.291
2|master|rpad len=10 N=100000000|333.304
2|v5|repeat len=100 N=1000000|0.229
2|v5b8m|repeat len=100 N=1000000|0.214
2|v5b8m-nont|repeat len=100 N=1000000|0.223
2|rfc|repeat len=100 N=1000000|0.266
2|master|repeat len=100 N=1000000|0.239
2|v5|rpad len=100 N=1000000|0.057
2|v5b8m|rpad len=100 N=1000000|0.052
2|v5b8m-nont|rpad len=100 N=1000000|0.049
2|rfc|rpad len=100 N=1000000|2.983
2|master|rpad len=100 N=1000000|2.924
2|v5|repeat len=100 N=16000000|5.575
2|v5b8m|repeat len=100 N=16000000|6.330
2|v5b8m-nont|repeat len=100 N=16000000|5.573
2|rfc|repeat len=100 N=16000000|6.332
2|master|repeat len=100 N=16000000|6.348
2|v5|rpad len=100 N=16000000|3.478
2|v5b8m|rpad len=100 N=16000000|3.847
2|v5b8m-nont|rpad len=100 N=16000000|4.249
2|rfc|rpad len=100 N=16000000|49.297
2|master|rpad len=100 N=16000000|50.337
2|v5|repeat len=100 N=100000000|47.476
2|v5b8m|repeat len=100 N=100000000|49.504
2|v5b8m-nont|repeat len=100 N=100000000|50.675
2|rfc|repeat len=100 N=100000000|48.258
2|master|repeat len=100 N=100000000|47.584
2|v5|rpad len=100 N=100000000|49.109
2|v5b8m|rpad len=100 N=100000000|50.932
2|v5b8m-nont|rpad len=100 N=100000000|53.914
2|rfc|rpad len=100 N=100000000|335.755
2|master|rpad len=100 N=100000000|325.102
3|v5b8m|repeat len=1 N=1000000|0.225
3|v5b8m-nont|repeat len=1 N=1000000|0.216
3|rfc|repeat len=1 N=1000000|0.204
3|master|repeat len=1 N=1000000|1.444
3|v5|repeat len=1 N=1000000|0.214
3|v5b8m|rpad len=1 N=1000000|0.045
3|v5b8m-nont|rpad len=1 N=1000000|0.045
3|rfc|rpad len=1 N=1000000|2.924
3|master|rpad len=1 N=1000000|2.949
3|v5|rpad len=1 N=1000000|0.039
3|v5b8m|repeat len=1 N=16000000|6.969
3|v5b8m-nont|repeat len=1 N=16000000|7.307
3|rfc|repeat len=1 N=16000000|7.080
3|master|repeat len=1 N=16000000|27.899
3|v5|repeat len=1 N=16000000|7.071
3|v5b8m|rpad len=1 N=16000000|4.553
3|v5b8m-nont|rpad len=1 N=16000000|4.437
3|rfc|rpad len=1 N=16000000|49.731
3|master|rpad len=1 N=16000000|49.445
3|v5|rpad len=1 N=16000000|4.248
3|v5b8m|repeat len=1 N=100000000|46.587
3|v5b8m-nont|repeat len=1 N=100000000|47.027
3|rfc|repeat len=1 N=100000000|49.059
3|master|repeat len=1 N=100000000|173.132
3|v5|repeat len=1 N=100000000|47.468
3|v5b8m|rpad len=1 N=100000000|48.186
3|v5b8m-nont|rpad len=1 N=100000000|48.658
3|rfc|rpad len=1 N=100000000|328.980
3|master|rpad len=1 N=100000000|327.377
3|v5|rpad len=1 N=100000000|46.348
3|v5b8m|repeat len=10 N=1000000|0.204
3|v5b8m-nont|repeat len=10 N=1000000|0.219
3|rfc|repeat len=10 N=1000000|0.320
3|master|repeat len=10 N=1000000|0.327
3|v5|repeat len=10 N=1000000|0.212
3|v5b8m|rpad len=10 N=1000000|0.040
3|v5b8m-nont|rpad len=10 N=1000000|0.041
3|rfc|rpad len=10 N=1000000|2.883
3|master|rpad len=10 N=1000000|2.942
3|v5|rpad len=10 N=1000000|0.052
3|v5b8m|repeat len=10 N=16000000|8.526
3|v5b8m-nont|repeat len=10 N=16000000|6.800
3|rfc|repeat len=10 N=16000000|9.973
3|master|repeat len=10 N=16000000|9.376
3|v5|repeat len=10 N=16000000|6.894
3|v5b8m|rpad len=10 N=16000000|4.485
3|v5b8m-nont|rpad len=10 N=16000000|4.624
3|rfc|rpad len=10 N=16000000|50.511
3|master|rpad len=10 N=16000000|50.804
3|v5|rpad len=10 N=16000000|4.238
3|v5b8m|repeat len=10 N=100000000|57.266
3|v5b8m-nont|repeat len=10 N=100000000|58.048
3|rfc|repeat len=10 N=100000000|65.732
3|master|repeat len=10 N=100000000|65.570
3|v5|repeat len=10 N=100000000|49.838
3|v5b8m|rpad len=10 N=100000000|48.594
3|v5b8m-nont|rpad len=10 N=100000000|46.646
3|rfc|rpad len=10 N=100000000|325.341
3|master|rpad len=10 N=100000000|342.535
3|v5|rpad len=10 N=100000000|45.261
3|v5b8m|repeat len=100 N=1000000|0.226
3|v5b8m-nont|repeat len=100 N=1000000|0.243
3|rfc|repeat len=100 N=1000000|0.248
3|master|repeat len=100 N=1000000|0.227
3|v5|repeat len=100 N=1000000|0.216
3|v5b8m|rpad len=100 N=1000000|0.040
3|v5b8m-nont|rpad len=100 N=1000000|0.045
3|rfc|rpad len=100 N=1000000|2.916
3|master|rpad len=100 N=1000000|2.929
3|v5|rpad len=100 N=1000000|0.041
3|v5b8m|repeat len=100 N=16000000|6.860
3|v5b8m-nont|repeat len=100 N=16000000|6.576
3|rfc|repeat len=100 N=16000000|6.948
3|master|repeat len=100 N=16000000|7.063
3|v5|repeat len=100 N=16000000|6.928
3|v5b8m|rpad len=100 N=16000000|4.148
3|v5b8m-nont|rpad len=100 N=16000000|4.610
3|rfc|rpad len=100 N=16000000|50.180
3|master|rpad len=100 N=16000000|51.905
3|v5|rpad len=100 N=16000000|4.713
3|v5b8m|repeat len=100 N=100000000|45.009
3|v5b8m-nont|repeat len=100 N=100000000|42.563
3|rfc|repeat len=100 N=100000000|40.962
3|master|repeat len=100 N=100000000|39.873
3|v5|repeat len=100 N=100000000|43.671
3|v5b8m|rpad len=100 N=100000000|46.044
3|v5b8m-nont|rpad len=100 N=100000000|50.311
3|rfc|rpad len=100 N=100000000|332.144
3|master|rpad len=100 N=100000000|325.854
3|v5|rpad len=100 N=100000000|41.055
4|v5b8m-nont|repeat len=1 N=1000000|0.208
4|rfc|repeat len=1 N=1000000|0.219
4|master|repeat len=1 N=1000000|1.416
4|v5|repeat len=1 N=1000000|0.209
4|v5b8m|repeat len=1 N=1000000|0.227
4|v5b8m-nont|rpad len=1 N=1000000|0.039
4|rfc|rpad len=1 N=1000000|3.046
4|master|rpad len=1 N=1000000|2.882
4|v5|rpad len=1 N=1000000|0.043
4|v5b8m|rpad len=1 N=1000000|0.042
4|v5b8m-nont|repeat len=1 N=16000000|6.569
4|rfc|repeat len=1 N=16000000|6.695
4|master|repeat len=1 N=16000000|27.279
4|v5|repeat len=1 N=16000000|6.337
4|v5b8m|repeat len=1 N=16000000|7.623
4|v5b8m-nont|rpad len=1 N=16000000|3.730
4|rfc|rpad len=1 N=16000000|52.228
4|master|rpad len=1 N=16000000|51.453
4|v5|rpad len=1 N=16000000|3.720
4|v5b8m|rpad len=1 N=16000000|4.211
4|v5b8m-nont|repeat len=1 N=100000000|45.719
4|rfc|repeat len=1 N=100000000|47.822
4|master|repeat len=1 N=100000000|174.628
4|v5|repeat len=1 N=100000000|40.432
4|v5b8m|repeat len=1 N=100000000|40.503
4|v5b8m-nont|rpad len=1 N=100000000|42.527
4|rfc|rpad len=1 N=100000000|343.643
4|master|rpad len=1 N=100000000|333.860
4|v5|rpad len=1 N=100000000|45.497
4|v5b8m|rpad len=1 N=100000000|42.479
4|v5b8m-nont|repeat len=10 N=1000000|0.202
4|rfc|repeat len=10 N=1000000|0.323
4|master|repeat len=10 N=1000000|0.362
4|v5|repeat len=10 N=1000000|0.240
4|v5b8m|repeat len=10 N=1000000|0.284
4|v5b8m-nont|rpad len=10 N=1000000|0.043
4|rfc|rpad len=10 N=1000000|3.224
4|master|rpad len=10 N=1000000|2.999
4|v5|rpad len=10 N=1000000|0.038
4|v5b8m|rpad len=10 N=1000000|0.043
4|v5b8m-nont|repeat len=10 N=16000000|6.239
4|rfc|repeat len=10 N=16000000|8.619
4|master|repeat len=10 N=16000000|8.480
4|v5|repeat len=10 N=16000000|6.071
4|v5b8m|repeat len=10 N=16000000|5.893
4|v5b8m-nont|rpad len=10 N=16000000|3.291
4|rfc|rpad len=10 N=16000000|51.833
4|master|rpad len=10 N=16000000|48.376
4|v5|rpad len=10 N=16000000|3.425
4|v5b8m|rpad len=10 N=16000000|4.464
4|v5b8m-nont|repeat len=10 N=100000000|43.100
4|rfc|repeat len=10 N=100000000|52.106
4|master|repeat len=10 N=100000000|52.496
4|v5|repeat len=10 N=100000000|43.130
4|v5b8m|repeat len=10 N=100000000|46.421
4|v5b8m-nont|rpad len=10 N=100000000|46.446
4|rfc|rpad len=10 N=100000000|334.879
4|master|rpad len=10 N=100000000|345.571
4|v5|rpad len=10 N=100000000|44.577
4|v5b8m|rpad len=10 N=100000000|46.815
4|v5b8m-nont|repeat len=100 N=1000000|0.216
4|rfc|repeat len=100 N=1000000|0.262
4|master|repeat len=100 N=1000000|0.255
4|v5|repeat len=100 N=1000000|0.254
4|v5b8m|repeat len=100 N=1000000|0.254
4|v5b8m-nont|rpad len=100 N=1000000|0.060
4|rfc|rpad len=100 N=1000000|3.073
4|master|rpad len=100 N=1000000|3.176
4|v5|rpad len=100 N=1000000|0.060
4|v5b8m|rpad len=100 N=1000000|0.057
4|v5b8m-nont|repeat len=100 N=16000000|6.675
4|rfc|repeat len=100 N=16000000|6.211
4|master|repeat len=100 N=16000000|5.921
4|v5|repeat len=100 N=16000000|6.251
4|v5b8m|repeat len=100 N=16000000|6.846
4|v5b8m-nont|rpad len=100 N=16000000|3.940
4|rfc|rpad len=100 N=16000000|51.230
4|master|rpad len=100 N=16000000|52.143
4|v5|rpad len=100 N=16000000|3.849
4|v5b8m|rpad len=100 N=16000000|4.169
4|v5b8m-nont|repeat len=100 N=100000000|48.362
4|rfc|repeat len=100 N=100000000|46.295
4|master|repeat len=100 N=100000000|49.466
4|v5|repeat len=100 N=100000000|44.867
4|v5b8m|repeat len=100 N=100000000|47.551
4|v5b8m-nont|rpad len=100 N=100000000|53.332
4|rfc|rpad len=100 N=100000000|339.208
4|master|rpad len=100 N=100000000|334.218
4|v5|rpad len=100 N=100000000|43.249
4|v5b8m|rpad len=100 N=100000000|51.759
5|rfc|repeat len=1 N=1000000|0.285
5|master|repeat len=1 N=1000000|1.496
5|v5|repeat len=1 N=1000000|0.239
5|v5b8m|repeat len=1 N=1000000|0.241
5|v5b8m-nont|repeat len=1 N=1000000|0.241
5|rfc|rpad len=1 N=1000000|3.045
5|master|rpad len=1 N=1000000|3.028
5|v5|rpad len=1 N=1000000|0.067
5|v5b8m|rpad len=1 N=1000000|0.041
5|v5b8m-nont|rpad len=1 N=1000000|0.050
5|rfc|repeat len=1 N=16000000|6.403
5|master|repeat len=1 N=16000000|27.261
5|v5|repeat len=1 N=16000000|6.948
5|v5b8m|repeat len=1 N=16000000|8.551
5|v5b8m-nont|repeat len=1 N=16000000|7.136
5|rfc|rpad len=1 N=16000000|50.064
5|master|rpad len=1 N=16000000|49.933
5|v5|rpad len=1 N=16000000|4.108
5|v5b8m|rpad len=1 N=16000000|3.908
5|v5b8m-nont|rpad len=1 N=16000000|3.902
5|rfc|repeat len=1 N=100000000|44.838
5|master|repeat len=1 N=100000000|175.551
5|v5|repeat len=1 N=100000000|54.855
5|v5b8m|repeat len=1 N=100000000|51.297
5|v5b8m-nont|repeat len=1 N=100000000|51.311
5|rfc|rpad len=1 N=100000000|330.721
5|master|rpad len=1 N=100000000|333.589
5|v5|rpad len=1 N=100000000|61.238
5|v5b8m|rpad len=1 N=100000000|49.432
5|v5b8m-nont|rpad len=1 N=100000000|48.070
5|rfc|repeat len=10 N=1000000|0.327
5|master|repeat len=10 N=1000000|0.326
5|v5|repeat len=10 N=1000000|0.280
5|v5b8m|repeat len=10 N=1000000|0.263
5|v5b8m-nont|repeat len=10 N=1000000|0.264
5|rfc|rpad len=10 N=1000000|3.169
5|master|rpad len=10 N=1000000|3.388
5|v5|rpad len=10 N=1000000|0.072
5|v5b8m|rpad len=10 N=1000000|0.062
5|v5b8m-nont|rpad len=10 N=1000000|0.058
5|rfc|repeat len=10 N=16000000|9.053
5|master|repeat len=10 N=16000000|9.326
5|v5|repeat len=10 N=16000000|6.838
5|v5b8m|repeat len=10 N=16000000|7.181
5|v5b8m-nont|repeat len=10 N=16000000|6.941
5|rfc|rpad len=10 N=16000000|51.329
5|master|rpad len=10 N=16000000|50.406
5|v5|rpad len=10 N=16000000|3.777
5|v5b8m|rpad len=10 N=16000000|3.994
5|v5b8m-nont|rpad len=10 N=16000000|3.771
5|rfc|repeat len=10 N=100000000|54.950
5|master|repeat len=10 N=100000000|60.661
5|v5|repeat len=10 N=100000000|46.783
5|v5b8m|repeat len=10 N=100000000|50.412
5|v5b8m-nont|repeat len=10 N=100000000|45.181
5|rfc|rpad len=10 N=100000000|325.700
5|master|rpad len=10 N=100000000|346.414
5|v5|rpad len=10 N=100000000|40.888
5|v5b8m|rpad len=10 N=100000000|42.669
5|v5b8m-nont|rpad len=10 N=100000000|42.978
5|rfc|repeat len=100 N=1000000|0.316
5|master|repeat len=100 N=1000000|0.271
5|v5|repeat len=100 N=1000000|0.256
5|v5b8m|repeat len=100 N=1000000|0.286
5|v5b8m-nont|repeat len=100 N=1000000|0.262
5|rfc|rpad len=100 N=1000000|2.935
5|master|rpad len=100 N=1000000|3.027
5|v5|rpad len=100 N=1000000|0.044
5|v5b8m|rpad len=100 N=1000000|0.044
5|v5b8m-nont|rpad len=100 N=1000000|0.068
5|rfc|repeat len=100 N=16000000|7.206
5|master|repeat len=100 N=16000000|6.253
5|v5|repeat len=100 N=16000000|5.819
5|v5b8m|repeat len=100 N=16000000|6.729
5|v5b8m-nont|repeat len=100 N=16000000|6.429
5|rfc|rpad len=100 N=16000000|50.828
5|master|rpad len=100 N=16000000|53.784
5|v5|rpad len=100 N=16000000|3.704
5|v5b8m|rpad len=100 N=16000000|3.630
5|v5b8m-nont|rpad len=100 N=16000000|3.441
5|rfc|repeat len=100 N=100000000|46.388
5|master|repeat len=100 N=100000000|46.266
5|v5|repeat len=100 N=100000000|45.845
5|v5b8m|repeat len=100 N=100000000|48.169
5|v5b8m-nont|repeat len=100 N=100000000|50.347
5|rfc|rpad len=100 N=100000000|332.292
5|master|rpad len=100 N=100000000|333.209
5|v5|rpad len=100 N=100000000|44.833
5|v5b8m|rpad len=100 N=100000000|48.334
5|v5b8m-nont|rpad len=100 N=100000000|50.560
===== diffcheck.sh =====
#!/bin/bash
# Differential check: the same lpad()/rpad()/repeat() calls on master and on
# a patched build, in UTF8, EUC_JP and SQL_ASCII databases. Each call prints
# its length in bytes and characters and the md5 of its result, or the error
# message. Any line that differs between the two builds is printed.
# diffcheck.sh [patched build] (default: v5)
set -u
A=$(cd "$(dirname "$0")" && pwd)
W=$HOME/pgpad
NEW=${1:-v5}
gen() { # the calls, one per line, as SELECT statements
python3 - <<'EOF'
pads = ["", " ", "0", "ab", "abc", "é", "aéb", "日本", "日a本b",
"abcdefghij" * 10, "é" * 7 + "x"]
bases = ["", "x", "café", "日本語テキスト"]
ns = [-1, 0, 1, 2, 3, 5, 7, 99, 100, 101,
16383, 16384, 16385, 16386, 32767, 32768, 32769, 50000,
8388607, 8388608, 8388609, 20000001]
def lit(s): return "'" + s.replace("'", "''") + "'"
for f in ("lpad", "rpad"):
for b in bases:
for p in pads:
for n in ns:
print(f"SELECT $l${f}({lit(b)},{n},{lit(p)[:24]})$l$, octet_length(r), char_length(r), md5(r) "
f"FROM (SELECT {f}({lit(b)}, {n}, {lit(p)}) r) s;")
print(f"SELECT $l${f} too large$l$, octet_length({f}('x', 1073741823, 'ab'));")
print(f"SELECT $l${f} too large mb$l$, octet_length({f}('x', 400000000, '日本'));")
for s in ["", "x", "ab", "é", "日本", "abcdefghij" * 10, "aéb"]:
for c in [-1, 0, 1, 2, 3, 16383, 16384, 16385, 100000, 8388608, 8388609, 20000001]:
print(f"SELECT $l$repeat({lit(s)[:24]},{c})$l$, octet_length(r), char_length(r), md5(r) "
f"FROM (SELECT repeat({lit(s)}, {c}) r) s;")
print("SELECT 'repeat too large', octet_length(repeat('ab', 600000000));")
EOF
}
run() { # build dbname -> output
local b=$1 db=$2
# the script is UTF-8; psql would otherwise read it in the database encoding
PGCLIENTENCODING=UTF8 $W/i-$b/bin/psql -X -qAt -h /tmp -p ${PORT[$b]} -U postgres -d $db -f $A/diffcheck.sql 2>&1 \
| sed 's/^psql:[^:]*:[0-9]*: //'
}
gen > $A/diffcheck.sql
declare -A PORT DIR
port=55490
for b in master $NEW; do
D=$(mktemp -d /tmp/claude-1000/dc.XXXX); DIR[$b]=$D; PORT[$b]=$port
$W/i-$b/bin/initdb -D $D -A trust --no-sync -U postgres -E UTF8 --locale=C >/dev/null
printf "port = $port\nunix_socket_directories = '/tmp'\n" >> $D/postgresql.conf
$W/i-$b/bin/pg_ctl -D $D -l $D/log -w start >/dev/null
$W/i-$b/bin/psql -X -q -h /tmp -p $port -U postgres \
-c "CREATE DATABASE eucjp ENCODING 'EUC_JP' TEMPLATE template0" \
-c "CREATE DATABASE sqlascii ENCODING 'SQL_ASCII' TEMPLATE template0"
port=$((port + 1))
done
total=0; bad=0
for db in postgres eucjp sqlascii; do
run master $db > $A/diffcheck.master.$db.out
run $NEW $db > $A/diffcheck.$NEW.$db.out
n=$(wc -l < $A/diffcheck.master.$db.out)
d=$(diff $A/diffcheck.master.$db.out $A/diffcheck.$NEW.$db.out | grep -c '^<')
echo "$db: $n calls, $d differ"
diff $A/diffcheck.master.$db.out $A/diffcheck.$NEW.$db.out | head -20
total=$((total + n)); bad=$((bad + d))
done
echo "total: $total calls, $bad differ between master and $NEW"
echo "errors seen (same on both unless listed above):"
cat $A/diffcheck.master.*.out | grep -E 'ERROR' | sed 's/^[^E]*ERROR/ERROR/' | sort | uniq -c
for b in master $NEW; do $W/i-$b/bin/pg_ctl -D ${DIR[$b]} -m fast -w stop >/dev/null; rm -rf ${DIR[$b]}; done
===== ./diffcheck.sh v5 =====
postgres: 2025 calls, 0 differ
eucjp: 2025 calls, 0 differ
sqlascii: 2025 calls, 0 differ
total: 6075 calls, 0 differ between master and v5
errors seen (same on both unless listed above):
16 ERROR: requested length too large
===== cancel.sh =====
#!/bin/bash
# How long a large lpad()/rpad()/repeat() runs past statement_timeout.
# On master, lpad() and rpad() have no CHECK_FOR_INTERRUPTS() in the padding
# loop (repeat() has one per copy); with v5 all three go through
# repeat_bytes(), which checks once per block.
# cancel.sh [build...] (default: master v5)
set -u
A=$(cd "$(dirname "$0")" && pwd)
W=$HOME/pgpad
for b in ${*:-master v5}; do
D=$(mktemp -d /tmp/claude-1000/cx.XXXX); P=55499
$W/i-$b/bin/initdb -D $D -A trust --no-sync -U postgres >/dev/null
printf "port = $P\nunix_socket_directories = '/tmp'\n" >> $D/postgresql.conf
taskset -c 0 $W/i-$b/bin/pg_ctl -D $D -l $D/log -w start >/dev/null
for sql in "rpad('x', 268000000, 'ab')" "lpad('x', 268000000, 'ab')" "rpad('x', 268000000, ' ')" "repeat('ab', 134000000)"; do
for i in 1 2 3; do
t=$(taskset -c 2 $W/i-$b/bin/psql -X -qAt -h /tmp -p $P -U postgres \
-c "SET statement_timeout = '20ms'" -c '\timing on' \
-c "SELECT octet_length($sql)" 2>&1 | tr '\n' ' ' \
| sed -E 's/.*(ERROR: +[a-z ]+).*Time: ([0-9.]+) ms.*/\2 ms (\1)/; s/^([0-9]+) Time: ([0-9.]+) ms.*/\2 ms (completed)/')
echo "$b | $sql | $t"
done
done
$W/i-$b/bin/pg_ctl -D $D -m fast -w stop >/dev/null; rm -rf $D
done | tee $A/cancel.txt
===== ./cancel.sh master v5 =====
master | rpad('x', 268000000, 'ab') | 926.001 ms (ERROR: canceling statement due to statement timeout )
master | rpad('x', 268000000, 'ab') | 894.644 ms (ERROR: canceling statement due to statement timeout )
master | rpad('x', 268000000, 'ab') | 904.163 ms (ERROR: canceling statement due to statement timeout )
master | lpad('x', 268000000, 'ab') | 887.355 ms (ERROR: canceling statement due to statement timeout )
master | lpad('x', 268000000, 'ab') | 895.831 ms (ERROR: canceling statement due to statement timeout )
master | lpad('x', 268000000, 'ab') | 894.029 ms (ERROR: canceling statement due to statement timeout )
master | rpad('x', 268000000, ' ') | 896.186 ms (ERROR: canceling statement due to statement timeout )
master | rpad('x', 268000000, ' ') | 908.862 ms (ERROR: canceling statement due to statement timeout )
master | rpad('x', 268000000, ' ') | 905.626 ms (ERROR: canceling statement due to statement timeout )
master | repeat('ab', 134000000) | 20.651 ms (ERROR: canceling statement due to statement timeout )
master | repeat('ab', 134000000) | 20.636 ms (ERROR: canceling statement due to statement timeout )
master | repeat('ab', 134000000) | 21.316 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, 'ab') | 24.980 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, 'ab') | 23.898 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, 'ab') | 23.576 ms (ERROR: canceling statement due to statement timeout )
v5 | lpad('x', 268000000, 'ab') | 23.567 ms (ERROR: canceling statement due to statement timeout )
v5 | lpad('x', 268000000, 'ab') | 23.636 ms (ERROR: canceling statement due to statement timeout )
v5 | lpad('x', 268000000, 'ab') | 23.604 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, ' ') | 23.223 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, ' ') | 23.370 ms (ERROR: canceling statement due to statement timeout )
v5 | rpad('x', 268000000, ' ') | 23.517 ms (ERROR: canceling statement due to statement timeout )
v5 | repeat('ab', 134000000) | 23.232 ms (ERROR: canceling statement due to statement timeout )
v5 | repeat('ab', 134000000) | 24.497 ms (ERROR: canceling statement due to statement timeout )
v5 | repeat('ab', 134000000) | 24.486 ms (ERROR: canceling statement due to statement timeout )
* Re: Speed up lpad() and rpad() for one-byte padding strings
@ 2026-10-03 05:27 shihao zhong <zhong950419@gmail.com>
parent: Manu <manuelreyesbravo@gmail.com>
0 siblings, 0 replies; 12+ messages in thread
From: shihao zhong @ 2026-10-03 05:27 UTC (permalink / raw)
To: Manu <manuelreyesbravo@gmail.com>; +Cc: Sehrope Sarkuni <sehrope@jackdb.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nathan Bossart <nathandbossart@gmail.com>; Jan Nidzwetzki <jan@planetscale.com>; ChenhuiMo <chenhuimo.mch@qq.com>; pgsql-hackers
Hi,
Manu wrote:
> I have no ARM machine to check whether that carries over to
> repeat_bytes().
I ran v5 on an Apple M2 Max. It does not carry over. repeat() with a
one byte source matches the memset() patch, and rpad() to 100M goes
from 304 ms to 5.5 ms.
For the short cases I counted instructions instead of time, by reading
proc_pid_rusage() for the backend around each statement. Two master
servers agree to within one instruction per call.
Two things are slower than master.
lpad(s, 7, '0') takes 772 instructions per call against 694 on master.
Padding costs more up to three characters and breaks even at four.
repeat() with a 1000 byte source and a 10M result runs fewer
instructions, 4.06M against 4.33M, but is 24% slower. So that one is
memory cost. Changing the block size does not help here.
Thanks,
Shihao
^ permalink raw reply [nested|flat] 12+ messages in thread
end of thread, other threads:[~2026-10-03 05:27 UTC | newest]
Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-09-23 14:10 Speed up lpad() and rpad() for one-byte padding strings Sehrope Sarkuni <sehrope@jackdb.com>
2026-09-23 14:41 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-23 17:09 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-23 17:19 ` Sehrope Sarkuni <sehrope@jackdb.com>
2026-09-23 22:51 ` Sehrope Sarkuni <sehrope@jackdb.com>
2026-09-24 11:20 ` Heikki Linnakangas <hlinnaka@iki.fi>
2026-09-24 23:54 ` Sehrope Sarkuni <sehrope@jackdb.com>
2026-09-25 01:24 ` Manu <manuelreyesbravo@gmail.com>
2026-10-03 05:27 ` shihao zhong <zhong950419@gmail.com>
2026-09-23 17:16 ` Sehrope Sarkuni <sehrope@jackdb.com>
2026-09-24 05:21 ` John Naylor <johncnaylorls@gmail.com>
2026-09-24 11:03 ` Sehrope Sarkuni <sehrope@jackdb.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox