agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedBUG #19600: pgcrypto crypt() can return text with invalid encoding
2+ messages / 2 participants
[nested] [flat]
* BUG #19600: pgcrypto crypt() can return text with invalid encoding
@ 2026-08-03 05:25 PG Bug reporting form <noreply@postgresql.org>
2026-08-03 15:26 ` Re: BUG #19600: pgcrypto crypt() can return text with invalid encoding Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 1 reply; 2+ messages in thread
From: PG Bug reporting form @ 2026-08-03 05:25 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: malis@pgrust.com
The following bug has been logged on the website:
Bug reference: 19600
Logged by: Michael Malis
Email address: malis@pgrust.com
PostgreSQL version: 19beta1
Operating system: Debian
Description:
pgcrypto's crypt() copies bytes from the caller-supplied setting/salt
verbatim into its text result without regard to character boundaries. When a
multibyte character straddles the fixed byte count that gets copied, the
result is a text value containing a truncated UTF-8 sequence.
That value is invalid in the database encoding. The server rejects the exact
same bytes when they are offered as input, yet it accepts them from crypt()
and lets them into a table, after which upper() fails on the stored value
and
the row cannot be restored from a dump.
This is the same class that was already accepted and fixed for pgcrypto's
other text-returning functions. The thread "Encoding protection for
pgcrypto"
added pg_verifymbstr() to pgp_sym_decrypt_text and pgp_pub_decrypt_text on
the
stated principle that "by default the PostgreSQL TEXT type should have the
same encoding as the database encoding". crypt() is the same class and was
NOT covered. Unlike those functions it has no bytea counterpart, so the
escape
hatch offered in that thread ("use pgp_*_decrypt_bytea") does not exist
here.
Reproducer
----------
Needs pgcrypto and a UTF8 database. Nothing here requires superuser; crypt()
is EXECUTE-able by PUBLIC.
CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- the input salt is VALID UTF-8: U+20AC then 'A' -> e2 82 ac 41
SELECT encode(E'€A'::bytea, 'hex');
-- crypt() returns text whose bytes are NOT valid UTF-8
SELECT encode(crypt('password', E'€A')::bytea, 'hex');
-- e282555a6f49796a2f48792f63 <-- begins e2 82 55
-- the server rejects those same bytes on input
SELECT convert_from(decode('e282', 'hex'), 'UTF8');
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82
-- yet the value stores fine ...
CREATE TEMP TABLE t (h text);
INSERT INTO t SELECT crypt('password', E'€A');
SELECT octet_length(h) FROM t; -- 13
-- ... and then breaks a core text function
SELECT upper(h) FROM t;
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
-- and the row does not survive dump/restore
\copy t TO '/tmp/crypt_invalid.txt'
CREATE TEMP TABLE t2 (h text);
\copy t2 FROM '/tmp/crypt_invalid.txt'
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
-- CONTEXT: COPY t2, line 1
Two controls, both of which pass, so this is the geometry and not an
artifact of the test:
-- the documented idiom cannot trigger it: gen_salt() emits ASCII only
SELECT convert_from(crypt('password', gen_salt('des'))::bytea, 'UTF8')
IS NOT NULL; -- t
-- a multibyte character PAST the 2-byte cut is fine
SELECT convert_from(crypt('password', E'AB€')::bytea, 'UTF8')
IS NOT NULL; -- t
Expected vs. actual
-------------------
- Expected: crypt() either rejects a setting it cannot represent in the
database encoding, or returns a value that is valid in that encoding —
the same contract pgp_sym_decrypt_text and pgp_pub_decrypt_text now
honour.
- Actual: crypt() returns text containing a truncated multibyte sequence.
The
value is storable, breaks upper(), and blocks restore.
Mechanism, with file:line
-------------------------
contrib/pgcrypto/pgcrypto.c, pg_crypt(): the result is built with
cres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);
...
res = cstring_to_text(cres);
PG_RETURN_TEXT_P(res);
and there is no encoding validation on that path. The string "verifymbstr"
does not appear anywhere in pgcrypto.c, on 18.3 or on master.
contrib/pgcrypto/crypt-des.c, the traditional-DES branch, copies a FIXED
two bytes of the setting into the output:
output[0] = setting[0];
output[1] = setting[1] ? setting[1] : output[0];
with no character-boundary awareness. Identical on master.
So the rule is deterministic rather than intermittent: for traditional DES
the output is invalid exactly when a multibyte character straddles byte
offset 2, i.e. begins at byte 0 or byte 1 of the setting. The same mechanism
exists at different cut points elsewhere — xdes copies 9 bytes
(strlcpy(output, setting, 10)), and crypt-md5.c re-emits the raw salt run
capped at 8 bytes.
For contrast, contrib/pgcrypto/pgp-pgsql.c DOES validate, at two sites:
pg_verifymbstr(VARDATA_ANY(res), VARSIZE_ANY_EXHDR(res), false);
Observed behaviour
------------------
Verified live on both versions, in a UTF8 database:
crypt('password', E'€A') -> e282555a6f49796a2f48792f63
convert_from(decode('e282','hex'),'UTF8') -> ERROR, 0xe2 0x82
INSERT into a text column -> succeeds, octet_length 13
upper(stored) -> ERROR, 0xe2 0x82 0x55
COPY out then COPY in -> ERROR, 0xe2 0x82 0x55, COPY t2 line 1
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: BUG #19600: pgcrypto crypt() can return text with invalid encoding
2026-08-03 05:25 BUG #19600: pgcrypto crypt() can return text with invalid encoding PG Bug reporting form <noreply@postgresql.org>
@ 2026-08-03 15:26 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
0 siblings, 0 replies; 2+ messages in thread
From: Andrey Rachitskiy @ 2026-08-03 15:26 UTC (permalink / raw)
To: malis@pgrust.com; pgsql-bugs@lists.postgresql.org
Hi, Michael!
Big thanks for the steady flow of bug reports. Thanks for caring about
every detail. Really impressive productivity, keep it up!
The attached patch adds pg_verifymbstr() in pg_crypt(). A regress case
based on the report is included. Non-UTF8 buildfarm animals skip that
file.
пн, 3 авг. 2026 г. в 16:51, PG Bug reporting form <noreply@postgresql.org>:
> The following bug has been logged on the website:
>
> Bug reference: 19600
> Logged by: Michael Malis
> Email address: malis@pgrust.com
> PostgreSQL version: 19beta1
> Operating system: Debian
> Description:
>
> pgcrypto's crypt() copies bytes from the caller-supplied setting/salt
> verbatim into its text result without regard to character boundaries. When
> a
> multibyte character straddles the fixed byte count that gets copied, the
> result is a text value containing a truncated UTF-8 sequence.
>
> That value is invalid in the database encoding. The server rejects the
> exact
> same bytes when they are offered as input, yet it accepts them from crypt()
> and lets them into a table, after which upper() fails on the stored value
> and
> the row cannot be restored from a dump.
>
> This is the same class that was already accepted and fixed for pgcrypto's
> other text-returning functions. The thread "Encoding protection for
> pgcrypto"
> added pg_verifymbstr() to pgp_sym_decrypt_text and pgp_pub_decrypt_text on
> the
> stated principle that "by default the PostgreSQL TEXT type should have the
> same encoding as the database encoding". crypt() is the same class and was
> NOT covered. Unlike those functions it has no bytea counterpart, so the
> escape
> hatch offered in that thread ("use pgp_*_decrypt_bytea") does not exist
> here.
>
> Reproducer
> ----------
> Needs pgcrypto and a UTF8 database. Nothing here requires superuser;
> crypt()
> is EXECUTE-able by PUBLIC.
>
> CREATE EXTENSION IF NOT EXISTS pgcrypto;
>
> -- the input salt is VALID UTF-8: U+20AC then 'A' -> e2 82 ac 41
> SELECT encode(E'€A'::bytea, 'hex');
>
> -- crypt() returns text whose bytes are NOT valid UTF-8
> SELECT encode(crypt('password', E'€A')::bytea, 'hex');
> -- e282555a6f49796a2f48792f63 <-- begins e2 82 55
>
> -- the server rejects those same bytes on input
> SELECT convert_from(decode('e282', 'hex'), 'UTF8');
> -- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82
>
> -- yet the value stores fine ...
> CREATE TEMP TABLE t (h text);
> INSERT INTO t SELECT crypt('password', E'€A');
> SELECT octet_length(h) FROM t; -- 13
>
> -- ... and then breaks a core text function
> SELECT upper(h) FROM t;
> -- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
>
> -- and the row does not survive dump/restore
> \copy t TO '/tmp/crypt_invalid.txt'
> CREATE TEMP TABLE t2 (h text);
> \copy t2 FROM '/tmp/crypt_invalid.txt'
> -- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
> -- CONTEXT: COPY t2, line 1
>
> Two controls, both of which pass, so this is the geometry and not an
> artifact of the test:
>
> -- the documented idiom cannot trigger it: gen_salt() emits ASCII only
> SELECT convert_from(crypt('password', gen_salt('des'))::bytea, 'UTF8')
> IS NOT NULL; -- t
>
> -- a multibyte character PAST the 2-byte cut is fine
> SELECT convert_from(crypt('password', E'AB€')::bytea, 'UTF8')
> IS NOT NULL; -- t
>
> Expected vs. actual
> -------------------
> - Expected: crypt() either rejects a setting it cannot represent in the
> database encoding, or returns a value that is valid in that encoding —
> the same contract pgp_sym_decrypt_text and pgp_pub_decrypt_text now
> honour.
> - Actual: crypt() returns text containing a truncated multibyte sequence.
> The
> value is storable, breaks upper(), and blocks restore.
>
> Mechanism, with file:line
> -------------------------
> contrib/pgcrypto/pgcrypto.c, pg_crypt(): the result is built with
>
> cres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);
> ...
> res = cstring_to_text(cres);
> PG_RETURN_TEXT_P(res);
>
> and there is no encoding validation on that path. The string "verifymbstr"
> does not appear anywhere in pgcrypto.c, on 18.3 or on master.
>
> contrib/pgcrypto/crypt-des.c, the traditional-DES branch, copies a FIXED
> two bytes of the setting into the output:
>
> output[0] = setting[0];
> output[1] = setting[1] ? setting[1] : output[0];
>
> with no character-boundary awareness. Identical on master.
>
> So the rule is deterministic rather than intermittent: for traditional DES
> the output is invalid exactly when a multibyte character straddles byte
> offset 2, i.e. begins at byte 0 or byte 1 of the setting. The same
> mechanism
> exists at different cut points elsewhere — xdes copies 9 bytes
> (strlcpy(output, setting, 10)), and crypt-md5.c re-emits the raw salt run
> capped at 8 bytes.
>
> For contrast, contrib/pgcrypto/pgp-pgsql.c DOES validate, at two sites:
>
> pg_verifymbstr(VARDATA_ANY(res), VARSIZE_ANY_EXHDR(res), false);
>
> Observed behaviour
> ------------------
> Verified live on both versions, in a UTF8 database:
> crypt('password', E'€A') -> e282555a6f49796a2f48792f63
> convert_from(decode('e282','hex'),'UTF8') -> ERROR, 0xe2 0x82
> INSERT into a text column -> succeeds, octet_length 13
> upper(stored) -> ERROR, 0xe2 0x82 0x55
> COPY out then COPY in -> ERROR, 0xe2 0x82 0x55, COPY t2 line 1
>
>
>
>
>
--
Regards,
Rachitskiy Andrey
Attachments:
[text/x-patch] 0001-pgcrypto-Reject-crypt-results-invalid-in-database-encoding-BUG-19600.patch (4.3K, ../../CAB8bMivmoRuUcNF1pEFx=Tr2Q_b31VwtJse4P22pTMgoG_uC1w@mail.gmail.com/3-0001-pgcrypto-Reject-crypt-results-invalid-in-database-encoding-BUG-19600.patch)
download | inline diff:
From 728d0540949070eb965d8b843d2f62ba3e02c716 Mon Sep 17 00:00:00 2001
From: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Date: Mon, 3 Aug 2026 19:15:15 +0500
Subject: [PATCH] pgcrypto: Reject crypt() results invalid in database encoding
crypt() copies a fixed number of caller-supplied salt bytes into its
text result. When a multibyte character straddles that cut, the result
contains a truncated sequence that is not valid in the database
encoding, yet can still be stored. Validate the result with
pg_verifymbstr(), as already done for pgp_*_decrypt_text.
A regress case based on the report is included.
Bug: #19600
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reported-by: Michael Malis <malis@pgrust.com>
Discussion: https://postgr.es/m/19600-e5fb479f50f5022d@postgresql.org
---
contrib/pgcrypto/Makefile | 2 +-
contrib/pgcrypto/expected/crypt-utf8.out | 8 ++++++++
contrib/pgcrypto/expected/crypt-utf8_1.out | 4 ++++
contrib/pgcrypto/meson.build | 1 +
contrib/pgcrypto/pgcrypto.c | 4 ++++
contrib/pgcrypto/sql/crypt-utf8.sql | 7 +++++++
6 files changed, 25 insertions(+), 1 deletion(-)
create mode 100644 contrib/pgcrypto/expected/crypt-utf8.out
create mode 100644 contrib/pgcrypto/expected/crypt-utf8_1.out
create mode 100644 contrib/pgcrypto/sql/crypt-utf8.sql
diff --git a/contrib/pgcrypto/Makefile b/contrib/pgcrypto/Makefile
index 17d2b0c5ed1..ca9f99a1b16 100644
--- a/contrib/pgcrypto/Makefile
+++ b/contrib/pgcrypto/Makefile
@@ -42,7 +42,7 @@ PGFILEDESC = "pgcrypto - cryptographic functions"
REGRESS = init md5 sha1 hmac-md5 hmac-sha1 blowfish rijndael \
sha2 des 3des cast5 \
- crypt-des crypt-md5 crypt-blowfish crypt-xdes \
+ crypt-des crypt-md5 crypt-blowfish crypt-xdes crypt-utf8 \
pgp-armor pgp-decrypt pgp-encrypt pgp-encrypt-md5 $(CF_PGP_TESTS) \
pgp-pubkey-decrypt pgp-pubkey-encrypt pgp-pubkey-session \
pgp-info crypt-shacrypt
diff --git a/contrib/pgcrypto/expected/crypt-utf8.out b/contrib/pgcrypto/expected/crypt-utf8.out
new file mode 100644
index 00000000000..7ce1b570ef9
--- /dev/null
+++ b/contrib/pgcrypto/expected/crypt-utf8.out
@@ -0,0 +1,8 @@
+/* Needs UTF8; skip otherwise (truncated multibyte salt copy). */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- Salt bytes e282ac41 (euro then A). DES copies only e282, which is invalid UTF8.
+SELECT crypt('password', convert_from(decode('e282ac41', 'hex'), 'utf8'));
+ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
diff --git a/contrib/pgcrypto/expected/crypt-utf8_1.out b/contrib/pgcrypto/expected/crypt-utf8_1.out
new file mode 100644
index 00000000000..5375d85fef6
--- /dev/null
+++ b/contrib/pgcrypto/expected/crypt-utf8_1.out
@@ -0,0 +1,4 @@
+/* Needs UTF8; skip otherwise (truncated multibyte salt copy). */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build
index 4f255c8cb05..678a403baa1 100644
--- a/contrib/pgcrypto/meson.build
+++ b/contrib/pgcrypto/meson.build
@@ -46,6 +46,7 @@ pgcrypto_regress = [
'crypt-md5',
'crypt-blowfish',
'crypt-xdes',
+ 'crypt-utf8',
'pgp-armor',
'pgp-decrypt',
'pgp-encrypt',
diff --git a/contrib/pgcrypto/pgcrypto.c b/contrib/pgcrypto/pgcrypto.c
index 9ecbbd2e2f8..8f822d683d1 100644
--- a/contrib/pgcrypto/pgcrypto.c
+++ b/contrib/pgcrypto/pgcrypto.c
@@ -33,6 +33,7 @@
#include <ctype.h>
+#include "mb/pg_wchar.h"
#include "parser/scansup.h"
#include "pgcrypto.h"
#include "px-crypt.h"
@@ -239,6 +240,9 @@ pg_crypt(PG_FUNCTION_ARGS)
pfree(resbuf);
+ /* Ensure text result is valid in the database encoding. */
+ pg_verifymbstr(VARDATA_ANY(res), VARSIZE_ANY_EXHDR(res), false);
+
PG_FREE_IF_COPY(arg0, 0);
PG_FREE_IF_COPY(arg1, 1);
diff --git a/contrib/pgcrypto/sql/crypt-utf8.sql b/contrib/pgcrypto/sql/crypt-utf8.sql
new file mode 100644
index 00000000000..b6a45aabbe5
--- /dev/null
+++ b/contrib/pgcrypto/sql/crypt-utf8.sql
@@ -0,0 +1,7 @@
+/* Needs UTF8; skip otherwise (truncated multibyte salt copy). */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- Salt bytes e282ac41 (euro then A). DES copies only e282, which is invalid UTF8.
+SELECT crypt('password', convert_from(decode('e282ac41', 'hex'), 'utf8'));
--
2.53.0
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-08-03 15:26 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-08-03 05:25 BUG #19600: pgcrypto crypt() can return text with invalid encoding PG Bug reporting form <noreply@postgresql.org>
2026-08-03 15:26 ` Andrey Rachitskiy <pl0h0yp1@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox