From: Daniel Gustafsson <daniel@yesql.se>
Content-Type: multipart/mixed;
	boundary="Apple-Mail=_89F25642-0D50-49C9-8B8D-454D3B44E3D7"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.80.82.1.1\))
Subject: Cryptohash OpenSSL error queue in FIPS enabled builds
Message-Id: <C89D932C-501E-4473-9750-638CFCD9095E@yesql.se>
Date: Fri, 22 Apr 2022 16:56:38 +0200
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Archived-At: <https://www.postgresql.org/message-id/C89D932C-501E-4473-9750-638CFCD9095E%40yesql.se>
Precedence: bulk


--Apple-Mail=_89F25642-0D50-49C9-8B8D-454D3B44E3D7
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

In trying out an OpenSSL 3.1 build with FIPS enabled I realized that our
cryptohash code had a small issue.  Calling a banned cipher generated =
two
different error messages interleaved:

  postgres=3D# select md5('foo');
  ERROR:  could not compute MD5 hash: unsupported
  postgres=3D# select md5('foo');
  ERROR:  could not compute MD5 hash: initialization error

It turns out that OpenSSL places two errors in the queue for this =
operation,
and we only consume one without clearing the queue in between, so we =
grab an
error from the previous run.

Consuming all (both) errors and creating a concatenated string seems =
overkill
as it would alter the API from a const error string to something that =
needs
freeing etc (also, very few OpenSSL consumers actually drain the queue, =
OpenSSL
themselves don't).  Skimming the OpenSSL code I was unable to find =
another
example of two errors generated.  The attached calls ERR_clear_error() =
as how
we do in libpq in order to avoid consuming earlier errors.

--
Daniel Gustafsson		https://vmware.com/


--Apple-Mail=_89F25642-0D50-49C9-8B8D-454D3B44E3D7
Content-Disposition: attachment;
	filename=v1-0001-Clear-the-OpenSSL-error-queue-before-cryptohash-o.patch
Content-Type: application/octet-stream;
	x-unix-mode=0644;
	name="v1-0001-Clear-the-OpenSSL-error-queue-before-cryptohash-o.patch"
Content-Transfer-Encoding: quoted-printable

=46rom=2085c7ca37b20202c2e7e143a25489e51719de7f5e=20Mon=20Sep=2017=20=
00:00:00=202001=0AFrom:=20Daniel=20Gustafsson=20<daniel@yesql.se>=0A=
Date:=20Fri,=2022=20Apr=202022=2014:56:33=20+0200=0ASubject:=20[PATCH=20=
v1]=20Clear=20the=20OpenSSL=20error=20queue=20before=20cryptohash=20=
operations=0A=0ASetting=20up=20an=20EVP=20context=20for=20ciphers=20=
banned=20under=20FIPS=20generate=0Atwo=20OpenSSL=20errors=20in=20the=20=
queue,=20and=20as=20we=20only=20consume=20one=20from=0Athe=20queue=20the=20=
other=20is=20at=20the=20head=20for=20the=20next=20invocation:=0A=0A=20=20=
postgres=3D#=20select=20md5('foo');=0A=20=20ERROR:=20=20could=20not=20=
compute=20MD5=20hash:=20unsupported=0A=20=20postgres=3D#=20select=20=
md5('foo');=0A=20=20ERROR:=20=20could=20not=20compute=20MD5=20hash:=20=
initialization=20error=0A=0AClearing=20the=20error=20queue=20when=20=
creating=20the=20context=20ensures=20that=0Awe=20don't=20pull=20in=20an=20=
error=20from=20an=20earlier=20operation.=0A---=0A=20=
src/common/cryptohash_openssl.c=20|=202=20++=0A=20=
src/common/hmac_openssl.c=20=20=20=20=20=20=20|=202=20++=0A=202=20files=20=
changed,=204=20insertions(+)=0A=0Adiff=20--git=20=
a/src/common/cryptohash_openssl.c=20b/src/common/cryptohash_openssl.c=0A=
index=206c98f1cf95..aef3ef0476=20100644=0A---=20=
a/src/common/cryptohash_openssl.c=0A+++=20=
b/src/common/cryptohash_openssl.c=0A@@=20-115,6=20+115,8=20@@=20=
pg_cryptohash_create(pg_cryptohash_type=20type)=0A=20=09ctx->error=20=3D=20=
PG_CRYPTOHASH_ERROR_NONE;=0A=20=09ctx->errreason=20=3D=20NULL;=0A=20=0A+=09=
ERR_clear_error();=0A+=0A=20=09/*=0A=20=09=20*=20Initialization=20takes=20=
care=20of=20assigning=20the=20correct=20type=20for=20OpenSSL.=0A=20=09=20=
*/=0Adiff=20--git=20a/src/common/hmac_openssl.c=20=
b/src/common/hmac_openssl.c=0Aindex=2044f36d51dc..5eda902024=20100644=0A=
---=20a/src/common/hmac_openssl.c=0A+++=20b/src/common/hmac_openssl.c=0A=
@@=20-106,6=20+106,8=20@@=20pg_hmac_create(pg_cryptohash_type=20type)=0A=20=
=09ctx->error=20=3D=20PG_HMAC_ERROR_NONE;=0A=20=09ctx->errreason=20=3D=20=
NULL;=0A=20=0A+=09ERR_clear_error();=0A+=0A=20=09/*=0A=20=09=20*=20=
Initialization=20takes=20care=20of=20assigning=20the=20correct=20type=20=
for=20OpenSSL.=0A=20=09=20*/=0A--=20=0A2.32.0=20(Apple=20Git-132)=0A=0A=

--Apple-Mail=_89F25642-0D50-49C9-8B8D-454D3B44E3D7--