public inbox for [email protected]
help / color / mirror / Atom feedFrom: Jelte Fennema-Nio <[email protected]>
To: Xuneng Zhou <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Jacob Champion <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Daniel Gustafsson <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: Nazir Bilal Yavuz <[email protected]>
Subject: Re: RFC: adding pytest as a supported test framework
Date: Sat, 27 Dec 2025 18:26:55 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CABPTF7VMmg3JaAGYNkmytMZotFp2Ojzb3MyLwGHuB4UWWB1_Bw@mail.gmail.com>
References: <CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com>
<CA+TgmoY4DRMVEV7CVo1D8p==ExsN69W89rF7-Ax4V=M9HGWL7g@mail.gmail.com>
<CA+TgmoZV8B35Mvx8DqHtJjHDQxscJhwOWzNWiPj+dsUhwaUfSg@mail.gmail.com>
<[email protected]>
<CAOYmi+k214g1tgREvQkUcr=OCiVKyaTDu1ja+OGAygG1y=jhPQ@mail.gmail.com>
<CAOYmi+nEqA2LmetcJKUDmctypPLLumkVwj3vQ3idYd8yAGza5Q@mail.gmail.com>
<CAOYmi+kebAt6wSX7ee0c0kMzV7r0hp93bAt10V5a88yHHUKwog@mail.gmail.com>
<CAOYmi+=CrBFBRX-foRRES2tx2wXBJJhbsJGjgFbWvPcmBJuK-Q@mail.gmail.com>
<[email protected]>
<[email protected]>
<g4gdtfedwwdgu5sbcopjt3djtqk6p2q7n5nymp7ppapfwukoyd@ev2v4jtsbure>
<[email protected]>
<CABPTF7VMmg3JaAGYNkmytMZotFp2Ojzb3MyLwGHuB4UWWB1_Bw@mail.gmail.com>
On Fri Dec 19, 2025 at 3:18 PM CET, Xuneng Zhou wrote:
> Thanks for working on this. I’ve done an initial review of patch 4 and
> here are some comments below.
Attached is a version where I addressed all of those comemnts (the few
that I didn't or did in non-obvious ways are discussed at the end). I
also made a lot of improvements to the patchset:
1. Includes infrastructure to create multiple Postgres clusters in a
single test or module.
2. Basic documentation for the pytest testing infrastructure.
3. Configure pytest and dependencies in pyproject.toml instead of a
requirements file.
4. Set up the environment using "uv"[1] if no pytest binary can be
found and uv is found.
5. Use INITDB_TEMPLATE to speed up cluster creation.
6. Don't enable fsync during tests to speed them up.
7. I added a patch where I've rewritten the libpq load_balance_hosts
tests in python as a validation that the new infrastructure and APIs
work well. (the perl ones were also written by me)
8. Postgres logs are now included as a separate pytest "section" in the
output.
9. The pgtap output now includes all of the pytest sections. For an
example of what the failure output looks like, take a look here[2]
I also *removed* a few things that Jacob had initially added:
1. The SCRAM based windows auth. I think it's a good idea, but it
doesn't work with INITDB_TEMPLATE. I think that logic should be made
part of a separate patchset that stops use trust auth when creating
the INITDB_TEMPLATE. That way also the perl tests can benefit from it.
So seems good to do, but separate from the whole pytest work.
2. I removed the current_windows_user() function. This was dead code (as
also written in Jacob's comment) and python has built-in ways to get the current user.
3. I removed the fancy missing/incorrect dependency detection script. I
think (as Jacob also suggested in his code comments) that
importorskip is a better fit for this. Especially since we only have
pytest as a dependency for the core framework, and only the
cryptography package for the ssl tests.
Finally, I prepared a PR for our images to include the pytest
dependencies, so in a future version of the patchset we don't need to
ad-hoc install the required packages.
IMO patch 4 now serves as a good enough central infrastructure base for
people to develop tests on top of (which would possibly add some more
infrastructure as needed).
In regards to your second message, related to consensus on the
necessity of the project: Yes, based on the in-person conversations
about this people are either in favor of a pytest based test framework,
or neutral. There were no strong objections. We were now at the point
where "someone" now actually needs to do the work of getting some decent
infrastructure in place. Which is what Jacob and me have been trying to
do.
[1]: https://github.com/astral-sh/uv
[2]: https://cirrus-ci.com/task/4805772426608640?logs=test_world#L16
[3]: https://github.com/anarazel/pg-vm-images/pull/130
> 4) Type conversion: timestamp/timestamptz conversion could be wrong
> datetime.datetime.fromisoformat for both timestamp and timestamptz.
I now configured datestyle to ISO and UTC in postgresql.conf. If that
turns out not to be enough at some point (e.g. because a test sets a
different datestyle), we can revisit this.
> 6) Logging config: log_connections = all seems wrong
> print("log_connections = all", file=f)
>
> I don't see an option "all" for this parameter
> https://postgresqlco.nf/doc/en/param/log_connections/
That's a new value since PG18:
https://postgresqlco.nf/doc/en/param/log_connections/18/
> 7) UX: error message handling and query attachment
> raise_error() builds message with primary + optional Query: ....
>
> Should we include SQLSTATE and severity in the message string by
> default, because it helps when reading CI logs.
I don't think adding this additional info helps much anymore, now that
the postgres logs (item 8) are part of the output too. So I left it like
this, and even removed the query from the error message. By only
including the actual error message it's easier to match on it for errors
that a test expects. Matching on the SQLSTATE can be also done by
matching on the error type.
Attachments:
[text/x-patch] v5-0001-meson-Include-TAP-tests-in-the-configuration-summ.patch (1.1K, 2-v5-0001-meson-Include-TAP-tests-in-the-configuration-summ.patch)
download | inline diff:
From 00ed77c794a7c626a46d05aa782e1c382e29da0e Mon Sep 17 00:00:00 2001
From: Jacob Champion <[email protected]>
Date: Fri, 5 Sep 2025 16:39:08 -0700
Subject: [PATCH v5 1/7] meson: Include TAP tests in the configuration summary
...to make it obvious when they've been enabled. prove is added to the
executables list for good measure.
TODO: does Autoconf need something similar?
Per complaint by Peter Eisentraut.
---
meson.build | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/meson.build b/meson.build
index d7c5193d4ce..551e27f5eb3 100644
--- a/meson.build
+++ b/meson.build
@@ -3981,6 +3981,7 @@ summary(
'bison': '@0@ @1@'.format(bison.full_path(), bison_version),
'dtrace': dtrace,
'flex': '@0@ @1@'.format(flex.full_path(), flex_version),
+ 'prove': prove,
},
section: 'Programs',
)
@@ -4017,3 +4018,11 @@ summary(
section: 'External libraries',
list_sep: ' ',
)
+
+summary(
+ {
+ 'tap': tap_tests_enabled,
+ },
+ section: 'Other features',
+ list_sep: ' ',
+)
base-commit: 36b8f4974a884a7206df97f37ea62d2adc0b77f0
--
2.52.0
[text/x-patch] v5-0002-Add-support-for-pytest-test-suites.patch (30.8K, 3-v5-0002-Add-support-for-pytest-test-suites.patch)
download | inline diff:
From eae9e70547c3e92b53f6028292f387a6e5dd38d6 Mon Sep 17 00:00:00 2001
From: Jacob Champion <[email protected]>
Date: Wed, 13 Aug 2025 10:58:56 -0700
Subject: [PATCH v5 2/7] Add support for pytest test suites
Specify --enable-pytest/-Dpytest=enabled at configure time. This
contains no Postgres test logic -- it is just a "vanilla" pytest
skeleton.
I've written a custom pgtap output plugin, used by the Meson mtest
runner, to fully control what we see during CI test failures. The
pytest-tap plugin would have been preferable, but it's now in
maintenance mode, and it has problems with accidentally suppressing
important collection failures.
TODOs:
- The Chocolatey CI setup is subpar. Need to find a way to bless the
dependencies in use rather than pulling from pip... or maybe that will
be done by the image baker.
Co-authored-by: Jelte Fennema-Nio <[email protected]>
---
.cirrus.tasks.yml | 37 +++++--
.gitignore | 4 +
configure | 166 +++++++++++++++++++++++++++++-
configure.ac | 29 +++++-
meson.build | 107 +++++++++++++++++++
meson_options.txt | 8 +-
pyproject.toml | 21 ++++
src/Makefile.global.in | 29 ++++++
src/makefiles/meson.build | 2 +
src/test/Makefile | 2 +-
src/test/meson.build | 1 +
src/test/pytest/Makefile | 20 ++++
src/test/pytest/README | 1 +
src/test/pytest/meson.build | 16 +++
src/test/pytest/pgtap.py | 198 ++++++++++++++++++++++++++++++++++++
src/tools/testwrap | 6 +-
16 files changed, 631 insertions(+), 16 deletions(-)
create mode 100644 pyproject.toml
create mode 100644 src/test/pytest/Makefile
create mode 100644 src/test/pytest/README
create mode 100644 src/test/pytest/meson.build
create mode 100644 src/test/pytest/pgtap.py
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 038d043d00e..a83acb39e97 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -21,7 +21,8 @@ env:
# target to test, for all but windows
CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS
- CHECKFLAGS: -Otarget
+ # TODO were we avoiding --keep-going on purpose?
+ CHECKFLAGS: -Otarget --keep-going
PROVE_FLAGS: --timer
# Build test dependencies as part of the build step, to see compiler
# errors/warnings in one place.
@@ -44,6 +45,7 @@ env:
-Dldap=enabled
-Dssl=openssl
-Dtap_tests=enabled
+ -Dpytest=enabled
-Dplperl=enabled
-Dplpython=enabled
-Ddocs=enabled
@@ -225,7 +227,9 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y \
+ py311-packaging \
+ py311-pytest
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -317,7 +321,10 @@ task:
-Dpam=enabled
setup_additional_packages_script: |
- #pkgin -y install ...
+ pkgin -y install \
+ py312-packaging \
+ py312-test
+ ln -s /usr/pkg/bin/pytest-3.12 /usr/pkg/bin/pytest
<<: *netbsd_task_template
- name: OpenBSD - Meson
@@ -337,7 +344,9 @@ task:
-Duuid=e2fs
setup_additional_packages_script: |
- #pkg_add -I ...
+ pkg_add -I \
+ py3-test \
+ py3-packaging
# Always core dump to ${CORE_DUMP_DIR}
set_core_dump_script: sysctl -w kern.nosuidcoredump=2
<<: *openbsd_task_template
@@ -496,8 +505,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ python3-pytest \
+ python3-packaging
matrix:
# SPECIAL:
@@ -521,14 +532,15 @@ task:
set -e
./configure \
--enable-cassert --enable-injection-points --enable-debug \
- --enable-tap-tests --enable-nls \
+ --enable-tap-tests --enable-pytest --enable-nls \
--with-segsize-blocks=6 \
--with-libnuma \
--with-liburing \
\
${LINUX_CONFIGURE_FEATURES} \
\
- CLANG="ccache clang"
+ CLANG="ccache clang" \
+ PYTEST="env LD_PRELOAD=/lib/x86_64-linux-gnu/libasan.so.8 pytest"
EOF
build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin"
upload_caches: ccache
@@ -665,6 +677,8 @@ task:
p5.34-io-tty
p5.34-ipc-run
python312
+ py312-packaging
+ py312-pytest
tcl
zstd
@@ -714,6 +728,7 @@ task:
sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST
# system python doesn't provide headers
sudo /opt/local/bin/port select python3 python312
+ sudo /opt/local/bin/port select pytest pytest312
# Make macports install visible for subsequent steps
echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV
upload_caches: macports
@@ -787,6 +802,8 @@ task:
-Dldap=enabled
-Dssl=openssl
-Dtap_tests=enabled
+ -Dpytest=enabled
+ -DPYTEST=c:\Windows\system32\config\systemprofile\AppData\Roaming\Python\Python310\Scripts\pytest.exe
-Dplperl=enabled
-Dplpython=enabled
@@ -795,8 +812,10 @@ task:
depends_on: SanityCheck
only_if: $CI_WINDOWS_ENABLED
+ # XXX Does Chocolatey really not have any Python package installers?
setup_additional_packages_script: |
REM choco install -y --no-progress ...
+ pip3 install --user packaging pytest
setup_hosts_file_script: |
echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
@@ -859,7 +878,7 @@ task:
folder: ${CCACHE_DIR}
setup_additional_packages_script: |
- REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ...
+ C:\msys64\usr\bin\pacman.exe -S --noconfirm mingw-w64-ucrt-x86_64-python-pytest
mingw_info_script: |
%BASH% -c "where gcc"
diff --git a/.gitignore b/.gitignore
index 4e911395fe3..a8c73bba9ba 100644
--- a/.gitignore
+++ b/.gitignore
@@ -31,6 +31,8 @@ win32ver.rc
*.exe
lib*dll.def
lib*.pc
+__pycache__/
+*.egg-info/
# Local excludes in root directory
/GNUmakefile
@@ -43,3 +45,5 @@ lib*.pc
/Release/
/tmp_install/
/portlock/
+/.venv/
+/uv.lock
diff --git a/configure b/configure
index 14ad0a5006f..f28db423cd8 100755
--- a/configure
+++ b/configure
@@ -630,6 +630,8 @@ vpath_build
PG_SYSROOT
PG_VERSION_NUM
LDFLAGS_EX_BE
+UV
+PYTEST
PROVE
DBTOEPUB
FOP
@@ -772,6 +774,7 @@ CFLAGS
CC
enable_injection_points
PG_TEST_EXTRA
+enable_pytest
enable_tap_tests
enable_dtrace
DTRACEFLAGS
@@ -850,6 +853,7 @@ enable_profiling
enable_coverage
enable_dtrace
enable_tap_tests
+enable_pytest
enable_injection_points
with_blocksize
with_segsize
@@ -1550,7 +1554,10 @@ Optional Features:
--enable-profiling build with profiling enabled
--enable-coverage build with coverage testing instrumentation
--enable-dtrace build with DTrace support
- --enable-tap-tests enable TAP tests (requires Perl and IPC::Run)
+ --enable-tap-tests enable (Perl-based) TAP tests (requires Perl and
+ IPC::Run)
+ --enable-pytest enable (Python-based) pytest suites (requires
+ Python)
--enable-injection-points
enable injection points (for testing)
--enable-depend turn on automatic dependency tracking
@@ -3632,7 +3639,7 @@ fi
#
-# TAP tests
+# Test frameworks
#
@@ -3660,6 +3667,32 @@ fi
+
+# Check whether --enable-pytest was given.
+if test "${enable_pytest+set}" = set; then :
+ enableval=$enable_pytest;
+ case $enableval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --enable-pytest option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ enable_pytest=no
+
+fi
+
+
+
+
+
+
#
# Injection points
#
@@ -19229,6 +19262,135 @@ $as_echo "$modulestderr" >&6; }
fi
fi
+if test "$enable_pytest" = yes; then
+ if test -z "$PYTEST"; then
+ for ac_prog in pytest py.test
+do
+ # Extract the first word of "$ac_prog", so it can be a program name with args.
+set dummy $ac_prog; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_PYTEST+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ case $PYTEST in
+ [\\/]* | ?:[\\/]*)
+ ac_cv_path_PYTEST="$PYTEST" # Let the user override the test with a path.
+ ;;
+ *)
+ as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+ IFS=$as_save_IFS
+ test -z "$as_dir" && as_dir=.
+ for ac_exec_ext in '' $ac_executable_extensions; do
+ if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+ ac_cv_path_PYTEST="$as_dir/$ac_word$ac_exec_ext"
+ $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+ break 2
+ fi
+done
+ done
+IFS=$as_save_IFS
+
+ ;;
+esac
+fi
+PYTEST=$ac_cv_path_PYTEST
+if test -n "$PYTEST"; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PYTEST" >&5
+$as_echo "$PYTEST" >&6; }
+else
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+
+ test -n "$PYTEST" && break
+done
+
+else
+ # Report the value of PYTEST in configure's output in all cases.
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for PYTEST" >&5
+$as_echo_n "checking for PYTEST... " >&6; }
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PYTEST" >&5
+$as_echo "$PYTEST" >&6; }
+fi
+
+ if test -z "$PYTEST"; then
+ # If pytest not found, try installing with uv
+ if test -z "$UV"; then
+ for ac_prog in uv
+do
+ # Extract the first word of "$ac_prog", so it can be a program name with args.
+set dummy $ac_prog; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if ${ac_cv_path_UV+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ case $UV in
+ [\\/]* | ?:[\\/]*)
+ ac_cv_path_UV="$UV" # Let the user override the test with a path.
+ ;;
+ *)
+ as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+ IFS=$as_save_IFS
+ test -z "$as_dir" && as_dir=.
+ for ac_exec_ext in '' $ac_executable_extensions; do
+ if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
+ ac_cv_path_UV="$as_dir/$ac_word$ac_exec_ext"
+ $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+ break 2
+ fi
+done
+ done
+IFS=$as_save_IFS
+
+ ;;
+esac
+fi
+UV=$ac_cv_path_UV
+if test -n "$UV"; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $UV" >&5
+$as_echo "$UV" >&6; }
+else
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+
+ test -n "$UV" && break
+done
+
+else
+ # Report the value of UV in configure's output in all cases.
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for UV" >&5
+$as_echo_n "checking for UV... " >&6; }
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $UV" >&5
+$as_echo "$UV" >&6; }
+fi
+
+ if test -n "$UV"; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether uv can install pytest dependencies" >&5
+$as_echo_n "checking whether uv can install pytest dependencies... " >&6; }
+ if "$UV" pip install "$srcdir" >&5 2>&1; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+ PYTEST="$UV run pytest"
+ else
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ as_fn_error $? "pytest not found and uv failed to install dependencies" "$LINENO" 5
+ fi
+ else
+ as_fn_error $? "pytest not found" "$LINENO" 5
+ fi
+ fi
+fi
+
# If compiler will take -Wl,--as-needed (or various platform-specific
# spellings thereof) then add that to LDFLAGS. This is much easier than
# trying to filter LIBS to the minimum for each executable.
diff --git a/configure.ac b/configure.ac
index 01b3bbc1be8..8226e2a1342 100644
--- a/configure.ac
+++ b/configure.ac
@@ -225,11 +225,16 @@ AC_SUBST(DTRACEFLAGS)])
AC_SUBST(enable_dtrace)
#
-# TAP tests
+# Test frameworks
#
PGAC_ARG_BOOL(enable, tap-tests, no,
- [enable TAP tests (requires Perl and IPC::Run)])
+ [enable (Perl-based) TAP tests (requires Perl and IPC::Run)])
AC_SUBST(enable_tap_tests)
+
+PGAC_ARG_BOOL(enable, pytest, no,
+ [enable (Python-based) pytest suites (requires Python)])
+AC_SUBST(enable_pytest)
+
AC_ARG_VAR(PG_TEST_EXTRA,
[enable selected extra tests (overridden at runtime by PG_TEST_EXTRA environment variable)])
@@ -2412,6 +2417,26 @@ if test "$enable_tap_tests" = yes; then
fi
fi
+if test "$enable_pytest" = yes; then
+ PGAC_PATH_PROGS(PYTEST, pytest py.test)
+ if test -z "$PYTEST"; then
+ # If pytest not found, try installing with uv
+ PGAC_PATH_PROGS(UV, uv)
+ if test -n "$UV"; then
+ AC_MSG_CHECKING([whether uv can install pytest dependencies])
+ if "$UV" pip install "$srcdir" >&AS_MESSAGE_LOG_FD 2>&1; then
+ AC_MSG_RESULT([yes])
+ PYTEST="$UV run pytest"
+ else
+ AC_MSG_RESULT([no])
+ AC_MSG_ERROR([pytest not found and uv failed to install dependencies])
+ fi
+ else
+ AC_MSG_ERROR([pytest not found])
+ fi
+ fi
+fi
+
# If compiler will take -Wl,--as-needed (or various platform-specific
# spellings thereof) then add that to LDFLAGS. This is much easier than
# trying to filter LIBS to the minimum for each executable.
diff --git a/meson.build b/meson.build
index 551e27f5eb3..2ec125116a2 100644
--- a/meson.build
+++ b/meson.build
@@ -1711,6 +1711,41 @@ endif
+###############################################################
+# Library: pytest
+###############################################################
+
+pytest_enabled = false
+pytest = not_found_dep
+uv = not_found_dep
+use_uv = false
+
+pytestopt = get_option('pytest')
+if not pytestopt.disabled()
+ pytest = find_program(get_option('PYTEST'), native: true, required: false)
+
+ # If pytest not found, try installing with uv
+ if not pytest.found()
+ uv = find_program('uv', native: true, required: false)
+ if uv.found()
+ message('Installing pytest dependencies with uv...')
+ uv_install = run_command(uv, 'pip', 'install', meson.project_source_root(), check: false)
+ if uv_install.returncode() == 0
+ use_uv = true
+ pytest_enabled = true
+ endif
+ endif
+ else
+ pytest_enabled = true
+ endif
+
+ if not pytest_enabled and pytestopt.enabled()
+ error('pytest not found')
+ endif
+endif
+
+
+
###############################################################
# Library: zstd
###############################################################
@@ -3808,6 +3843,76 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ testwrap_pytest = testwrap_base
+ if not pytest_enabled
+ testwrap_pytest += ['--skip', 'pytest not enabled']
+ endif
+
+ if use_uv
+ test_command = [uv.full_path(), 'run', 'pytest']
+ elif pytest_enabled
+ test_command = [pytest.full_path()]
+ else
+ # Dummy value - test will be skipped anyway
+ test_command = ['pytest']
+ endif
+ test_command += [
+ '-c', meson.project_source_root() / 'pyproject.toml',
+ '--verbose',
+ '-p', 'pgtap', # enable our test reporter plugin
+ '-ra', # show skipped and xfailed tests too
+ ]
+
+ # Add temporary install, the build directory for non-installed binaries and
+ # also test/ for non-installed test binaries built separately.
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'], test_dir['bd'] / 'test')
+ temp_install_datadir = '@0@@1@'.format(test_install_destdir, dir_prefix / dir_data)
+ env.set('share_contrib_dir', temp_install_datadir / 'contrib')
+ # We also configure the same PYTHONPATH in the pytest settings in
+ # pyproject.toml, but pytest versions below 8.4 only actually use that
+ # value after plugin loading. So we need to configure it here too. This
+ # won't help people manually running pytest outside of meson/make, but we
+ # expect those to use a recent enough version of pytest anyway (and if
+ # not they can manually configure PYTHONPATH too).
+ env.prepend('PYTHONPATH', meson.project_source_root() / 'src' / 'test' / 'pytest')
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ test_group = test_dir['name']
+ test_kwargs = {
+ 'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps + t.get('deps', []),
+ 'env': env,
+ } + t.get('test_kwargs', {})
+
+ foreach onetest : t['tests']
+ # Make test names prettier, remove pyt/ and .py
+ onetest_p = onetest
+ if onetest_p.startswith('pyt/')
+ onetest_p = onetest.split('pyt/')[1]
+ endif
+ if onetest_p.endswith('.py')
+ onetest_p = fs.stem(onetest_p)
+ endif
+
+ test(test_dir['name'] / onetest_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_pytest + [
+ '--testgroup', test_dir['name'],
+ '--testname', onetest_p,
+ '--', test_command,
+ test_dir['sd'] / onetest,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
@@ -3982,6 +4087,7 @@ summary(
'dtrace': dtrace,
'flex': '@0@ @1@'.format(flex.full_path(), flex_version),
'prove': prove,
+ 'pytest': pytest,
},
section: 'Programs',
)
@@ -4022,6 +4128,7 @@ summary(
summary(
{
'tap': tap_tests_enabled,
+ 'pytest': pytest_enabled,
},
section: 'Other features',
list_sep: ' ',
diff --git a/meson_options.txt b/meson_options.txt
index 06bf5627d3c..88f22e699d9 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -41,7 +41,10 @@ option('cassert', type: 'boolean', value: false,
description: 'Enable assertion checks (for debugging)')
option('tap_tests', type: 'feature', value: 'auto',
- description: 'Enable TAP tests')
+ description: 'Enable (Perl-based) TAP tests')
+
+option('pytest', type: 'feature', value: 'auto',
+ description: 'Enable (Python-based) pytest suites')
option('injection_points', type: 'boolean', value: false,
description: 'Enable injection points')
@@ -195,6 +198,9 @@ option('PERL', type: 'string', value: 'perl',
option('PROVE', type: 'string', value: 'prove',
description: 'Path to prove binary')
+option('PYTEST', type: 'array', value: ['pytest', 'py.test'],
+ description: 'Path to pytest binary')
+
option('PYTHON', type: 'array', value: ['python3', 'python'],
description: 'Path to python binary')
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 00000000000..60abb4d0655
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,21 @@
+[project]
+name = "postgresql-hackers-tooling"
+version = "0.1.0"
+description = "Pytest infrastructure for PostgreSQL"
+requires-python = ">=3.6"
+dependencies = [
+ # pytest 7.0 was the last version which supported Python 3.6, but the BSDs
+ # have started putting 8.x into ports, so we support both. (pytest 8 can be
+ # used throughout once we drop support for Python 3.7.)
+ "pytest >= 7.0, < 10",
+
+ # Any other dependencies are effectively optional (added below). We import
+ # these libraries using pytest.importorskip(). So tests will be skipped if
+ # they are not available.
+]
+
+[tool.pytest.ini_options]
+minversion = "7.0"
+
+# Common test code can be found here.
+pythonpath = ["src/test/pytest"]
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 371cd7eba2c..160cdffd4f1 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -211,6 +211,7 @@ enable_dtrace = @enable_dtrace@
enable_coverage = @enable_coverage@
enable_injection_points = @enable_injection_points@
enable_tap_tests = @enable_tap_tests@
+enable_pytest = @enable_pytest@
python_includespec = @python_includespec@
python_libdir = @python_libdir@
@@ -354,6 +355,7 @@ MSGFMT = @MSGFMT@
MSGFMT_FLAGS = @MSGFMT_FLAGS@
MSGMERGE = @MSGMERGE@
OPENSSL = @OPENSSL@
+PYTEST = @PYTEST@
PYTHON = @PYTHON@
TAR = @TAR@
XGETTEXT = @XGETTEXT@
@@ -508,6 +510,33 @@ prove_installcheck = @echo "TAP tests not enabled. Try configuring with --enable
prove_check = $(prove_installcheck)
endif
+ifeq ($(enable_pytest),yes)
+
+pytest_installcheck = @echo "Installcheck is not currently supported for pytest."
+
+# We also configure the same PYTHONPATH in the pytest settings in
+# pyproject.toml, but pytest versions below 8.4 only actually use that value
+# after plugin loading. So we need to configure it here too. This won't help
+# people manually running pytest outside of meson/make, but we expect those to
+# use a recent enough version of pytest anyway (and if not they can manually
+# configure PYTHONPATH too).
+define pytest_check
+echo "# +++ pytest check in $(subdir) +++" && \
+rm -rf '$(CURDIR)'/tmp_check && \
+$(MKDIR_P) '$(CURDIR)'/tmp_check && \
+cd $(srcdir) && \
+ TESTLOGDIR='$(CURDIR)/tmp_check/log' \
+ TESTDATADIR='$(CURDIR)/tmp_check' \
+ PYTHONPATH='$(abs_top_srcdir)/src/test/pytest:$$PYTHONPATH' \
+ $(with_temp_install) \
+ $(PYTEST) -c '$(abs_top_srcdir)/pyproject.toml' --verbose -ra ./pyt/
+endef
+
+else
+pytest_installcheck = @echo "pytest is not enabled. Try configuring with --enable-pytest"
+pytest_check = $(pytest_installcheck)
+endif
+
# Installation.
install_bin = @install_bin@
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index c6edf14ec44..5b9a804aa94 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -56,6 +56,7 @@ pgxs_kv = {
'enable_nls': libintl.found() ? 'yes' : 'no',
'enable_injection_points': get_option('injection_points') ? 'yes' : 'no',
'enable_tap_tests': tap_tests_enabled ? 'yes' : 'no',
+ 'enable_pytest': pytest_enabled ? 'yes' : 'no',
'enable_debug': get_option('debug') ? 'yes' : 'no',
'enable_coverage': 'no',
'enable_dtrace': dtrace.found() ? 'yes' : 'no',
@@ -145,6 +146,7 @@ pgxs_bins = {
'OPENSSL': openssl,
'PERL': perl,
'PROVE': prove,
+ 'PYTEST': pytest,
'PYTHON': python,
'TAR': tar,
'ZSTD': program_zstd,
diff --git a/src/test/Makefile b/src/test/Makefile
index 511a72e6238..c035dbb7fc7 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,7 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl postmaster regress isolation modules authentication recovery subscription
+SUBDIRS = perl postmaster pytest regress isolation modules authentication recovery subscription
ifeq ($(with_icu),yes)
SUBDIRS += icu
diff --git a/src/test/meson.build b/src/test/meson.build
index ccc31d6a86a..d08a6ef61c2 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -5,6 +5,7 @@ subdir('isolation')
subdir('authentication')
subdir('postmaster')
+subdir('pytest')
subdir('recovery')
subdir('subscription')
subdir('modules')
diff --git a/src/test/pytest/Makefile b/src/test/pytest/Makefile
new file mode 100644
index 00000000000..2bdca96ccbe
--- /dev/null
+++ b/src/test/pytest/Makefile
@@ -0,0 +1,20 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for pytest
+#
+# Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/pytest/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/pytest
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+check:
+ $(pytest_check)
+
+clean distclean maintainer-clean:
+ rm -rf tmp_check
diff --git a/src/test/pytest/README b/src/test/pytest/README
new file mode 100644
index 00000000000..1333ed77b7e
--- /dev/null
+++ b/src/test/pytest/README
@@ -0,0 +1 @@
+TODO
diff --git a/src/test/pytest/meson.build b/src/test/pytest/meson.build
new file mode 100644
index 00000000000..abd128dfa24
--- /dev/null
+++ b/src/test/pytest/meson.build
@@ -0,0 +1,16 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+if not pytest_enabled
+ subdir_done()
+endif
+
+tests += {
+ 'name': 'pytest',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'tests': [
+ 'pyt/test_something.py',
+ ],
+ },
+}
diff --git a/src/test/pytest/pgtap.py b/src/test/pytest/pgtap.py
new file mode 100644
index 00000000000..c92cad98d95
--- /dev/null
+++ b/src/test/pytest/pgtap.py
@@ -0,0 +1,198 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import os
+import sys
+
+import pytest
+
+#
+# Helpers
+#
+
+
+class TAP:
+ """
+ A basic API for reporting via the TAP protocol.
+ """
+
+ def __init__(self):
+ self.count = 0
+
+ # XXX interacts poorly with testwrap's boilerplate diagnostics
+ # self.print("TAP version 13")
+
+ def expect(self, num: int):
+ self.print(f"1..{num}")
+
+ def print(self, *args):
+ print(*args, file=sys.__stdout__)
+
+ def ok(self, name: str):
+ self.count += 1
+ self.print("ok", self.count, "-", name)
+
+ def skip(self, name: str, reason: str):
+ self.count += 1
+ self.print("ok", self.count, "-", name, "# skip", reason)
+
+ def fail(self, name: str, details: str):
+ self.count += 1
+ self.print("not ok", self.count, "-", name)
+
+ # mtest has some odd behavior around TAP tests where it won't print
+ # diagnostics on failure if they're part of the stdout stream, so we
+ # might as well just dump the details directly to stderr instead.
+ print(details, file=sys.__stderr__)
+
+
+tap = TAP()
+
+
+class TestNotes:
+ """
+ Annotations for a single test. The existing pytest hooks keep interesting
+ information somewhat separated across the different stages
+ (setup/test/teardown), so this class is used to correlate them.
+ """
+
+ skipped = False
+ skip_reason = None
+
+ failed = False
+ details = ""
+
+
+# Register a custom key in the stash dictionary for keeping our TestNotes.
+notes_key = pytest.StashKey[TestNotes]()
+
+
+#
+# Hook Implementations
+#
+
+
[email protected](tryfirst=True)
+def pytest_configure(config):
+ """
+ Hijacks the standard streams as soon as possible during pytest startup. The
+ pytest-formatted output gets logged to file instead, and we'll use the
+ original sys.__stdout__/__stderr__ streams for the TAP protocol.
+ """
+ logdir = os.getenv("TESTLOGDIR")
+ if not logdir:
+ raise RuntimeError("pgtap requires the TESTLOGDIR envvar to be set")
+
+ os.makedirs(logdir)
+ logpath = os.path.join(logdir, "pytest.log")
+ sys.stdout = sys.stderr = open(logpath, "a", buffering=1)
+
+
[email protected](trylast=True)
+def pytest_sessionfinish(session, exitstatus):
+ """
+ Suppresses nonzero exit codes due to failed tests. (In that case, we want
+ Meson to report a failure count, not a generic ERROR.)
+ """
+ if exitstatus == pytest.ExitCode.TESTS_FAILED:
+ session.exitstatus = pytest.ExitCode.OK
+
+
[email protected]
+def pytest_collectreport(report):
+ # Include collection failures directly in Meson error output.
+ if report.failed:
+ print(report.longreprtext, file=sys.__stderr__)
+
+
[email protected]
+def pytest_internalerror(excrepr, excinfo):
+ # Include internal errors directly in Meson error output.
+ print(excrepr, file=sys.__stderr__)
+
+
+#
+# Hook Wrappers
+#
+# In pytest parlance, a "wrapper" for a hook can inspect and optionally modify
+# existing hooks' behavior, but it does not replace the hook chain. This is done
+# through a generator-style API which chains the hooks together (see the use of
+# `yield`).
+#
+
+
[email protected](hookwrapper=True)
+def pytest_collection(session):
+ """Reports the number of gathered tests after collection is finished."""
+ res = yield
+ tap.expect(session.testscollected)
+ return res
+
+
[email protected](hookwrapper=True)
+def pytest_runtest_makereport(item, call):
+ """
+ Annotates a test item with our TestNotes and grabs relevant information for
+ reporting.
+
+ This is called multiple times per test, so it's not correct to print the TAP
+ result here. (A test and its teardown stage can both fail, and we want to
+ see the details for both.) We instead combine all the information for use by
+ our pytest_runtest_protocol wrapper later on.
+ """
+ res = yield
+
+ if notes_key not in item.stash:
+ item.stash[notes_key] = TestNotes()
+ notes = item.stash[notes_key]
+
+ report = res.get_result()
+ if report.passed:
+ pass # no annotation needed
+
+ elif report.skipped:
+ notes.skipped = True
+ _, _, notes.skip_reason = report.longrepr
+
+ elif report.failed:
+ notes.failed = True
+
+ if not notes.details:
+ notes.details += "{:_^72}\n\n".format(f" {report.head_line} ")
+
+ if report.when in ("setup", "teardown"):
+ notes.details += "\n{:_^72}\n\n".format(
+ f" Error during {report.when} of {report.head_line} "
+ )
+
+ notes.details += report.longreprtext + "\n"
+
+ # Include captured stdout/stderr/log in failure output
+ for section_name, section_content in report.sections:
+ if section_content.strip():
+ notes.details += "\n{:-^72}\n".format(f" {section_name} ")
+ notes.details += section_content + "\n"
+
+ else:
+ raise RuntimeError("pytest_runtest_makereport received unknown test status")
+
+ return res
+
+
[email protected](hookwrapper=True)
+def pytest_runtest_protocol(item, nextitem):
+ """
+ Reports the TAP result for this test item using our gathered TestNotes.
+ """
+ res = yield
+
+ assert notes_key in item.stash, "pgtap didn't annotate a test item?"
+ notes = item.stash[notes_key]
+
+ if notes.failed:
+ tap.fail(item.nodeid, notes.details)
+ elif notes.skipped:
+ tap.skip(item.nodeid, notes.skip_reason)
+ else:
+ tap.ok(item.nodeid)
+
+ return res
diff --git a/src/tools/testwrap b/src/tools/testwrap
index e91296ecd15..346f86b8ea3 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -42,7 +42,11 @@ open(os.path.join(testdir, 'test.start'), 'x')
env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
- 'TESTLOGDIR': os.path.join(testdir, 'log')}
+ 'TESTLOGDIR': os.path.join(testdir, 'log'),
+ # Prevent emitting terminal capability sequences that pollute the
+ # TAP output stream (i.e.\033[?1034h). This happens on OpenBSD with
+ # pytest for unknown reasons.
+ 'TERM': ''}
# The configuration time value of PG_TEST_EXTRA is supplied via argument
--
2.52.0
[text/x-patch] v5-0003-ci-Add-MTEST_SUITES-for-optional-test-tailoring.patch (3.1K, 4-v5-0003-ci-Add-MTEST_SUITES-for-optional-test-tailoring.patch)
download | inline diff:
From 313ffe863b8a3eaf48ea578fc17596ad7078595f Mon Sep 17 00:00:00 2001
From: Jacob Champion <[email protected]>
Date: Tue, 2 Sep 2025 15:37:53 -0700
Subject: [PATCH v5 3/7] ci: Add MTEST_SUITES for optional test tailoring
Should make it easier to control the test cycle time for Cirrus. Add the
desired suites (remembering `--suite setup`!) to the top-level envvar.
---
.cirrus.tasks.yml | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index a83acb39e97..a2c3febc30c 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -28,6 +28,7 @@ env:
# errors/warnings in one place.
MBUILD_TARGET: all testprep
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+ MTEST_SUITES: # --suite setup --suite ssl --suite ...
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth
@@ -251,7 +252,7 @@ task:
su postgres <<-EOF
set -e
ulimit -c unlimited
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+ meson test $MTEST_ARGS --num-processes ${TEST_JOBS} ${MTEST_SUITES}
EOF
# test runningcheck, freebsd chosen because it's currently fast enough
@@ -396,7 +397,7 @@ task:
# Otherwise tests will fail on OpenBSD, due to inability to start enough
# processes.
ulimit -p 256
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+ meson test $MTEST_ARGS --num-processes ${TEST_JOBS} ${MTEST_SUITES}
EOF
on_failure:
@@ -614,7 +615,7 @@ task:
su postgres <<-EOF
set -e
ulimit -c unlimited
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+ meson test $MTEST_ARGS --num-processes ${TEST_JOBS} ${MTEST_SUITES}
EOF
# so that we don't upload 64bit logs if 32bit fails
rm -rf build/
@@ -627,7 +628,7 @@ task:
su postgres <<-EOF
set -e
ulimit -c unlimited
- PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
+ PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} ${MTEST_SUITES}
EOF
on_failure:
@@ -751,7 +752,7 @@ task:
test_world_script: |
ulimit -c unlimited # default is 0
ulimit -n 1024 # default is 256, pretty low
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+ meson test $MTEST_ARGS --num-processes ${TEST_JOBS} ${MTEST_SUITES}
on_failure:
<<: *on_failure_meson
@@ -834,7 +835,7 @@ task:
check_world_script: |
vcvarsall x64
- meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
+ meson test %MTEST_ARGS% --num-processes %TEST_JOBS% %MTEST_SUITES%
on_failure:
<<: *on_failure_meson
@@ -895,7 +896,7 @@ task:
upload_caches: ccache
test_world_script: |
- %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%"
+ %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS% %MTEST_SUITES%"
on_failure:
<<: *on_failure_meson
--
2.52.0
[text/x-patch] v5-0004-Add-pytest-infrastructure-to-interact-with-Postgr.patch (134.4K, 5-v5-0004-Add-pytest-infrastructure-to-interact-with-Postgr.patch)
download | inline diff:
From b710868bcb844416dc87faa70daab055a5cdb7f3 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Tue, 16 Dec 2025 09:25:48 +0100
Subject: [PATCH v5 4/7] Add pytest infrastructure to interact with PostgreSQL
servers
This adds functionality to the pytest infrastructure that allows tests
to do common things with PostgreSQL servers like:
- creating
- starting
- stopping
- connecting
- running queries
- handling errors
The goal of this infrastructure is to be so easy to use that the actual
tests really only contain the logic to test the behaviour that the tests
are testing, as opposed to a bunch of boilerplate. Examples of this are:
Types get converted to their Python counter parts automatically. Errors
become actual Python exceptions. Results of queries that only return a
single row or cell are unpacked automatically, so you don't have to do
rows[0][0] if the query only returns a single cell.
The only new tests that are part of this commit are tests that cover
this testing infrastructure itself. It's debatable whether such tests
are useful long term, because any infrastructure that's unused by actual
tests should probably not exist. For now it seems good to test this
basic functionality though, both to make sure we don't break it before
committing actual tests that use it, and also as an example for people
writing new tests.
---
doc/src/sgml/regress.sgml | 54 +-
pyproject.toml | 3 +
src/backend/utils/errcodes.txt | 5 +
src/test/pytest/README | 140 +-
src/test/pytest/libpq/__init__.py | 36 +
src/test/pytest/libpq/_core.py | 489 +++++
src/test/pytest/libpq/_error_base.py | 74 +
src/test/pytest/libpq/_generated_errors.py | 2116 ++++++++++++++++++++
src/test/pytest/libpq/errors.py | 39 +
src/test/pytest/meson.build | 5 +-
src/test/pytest/pypg/__init__.py | 10 +
src/test/pytest/pypg/_env.py | 72 +
src/test/pytest/pypg/fixtures.py | 335 ++++
src/test/pytest/pypg/server.py | 470 +++++
src/test/pytest/pypg/util.py | 42 +
src/test/pytest/pyt/conftest.py | 1 +
src/test/pytest/pyt/test_errors.py | 34 +
src/test/pytest/pyt/test_libpq.py | 172 ++
src/test/pytest/pyt/test_multi_server.py | 46 +
src/test/pytest/pyt/test_query_helpers.py | 347 ++++
src/tools/generate_pytest_libpq_errors.py | 147 ++
21 files changed, 4634 insertions(+), 3 deletions(-)
create mode 100644 src/test/pytest/libpq/__init__.py
create mode 100644 src/test/pytest/libpq/_core.py
create mode 100644 src/test/pytest/libpq/_error_base.py
create mode 100644 src/test/pytest/libpq/_generated_errors.py
create mode 100644 src/test/pytest/libpq/errors.py
create mode 100644 src/test/pytest/pypg/__init__.py
create mode 100644 src/test/pytest/pypg/_env.py
create mode 100644 src/test/pytest/pypg/fixtures.py
create mode 100644 src/test/pytest/pypg/server.py
create mode 100644 src/test/pytest/pypg/util.py
create mode 100644 src/test/pytest/pyt/conftest.py
create mode 100644 src/test/pytest/pyt/test_errors.py
create mode 100644 src/test/pytest/pyt/test_libpq.py
create mode 100644 src/test/pytest/pyt/test_multi_server.py
create mode 100644 src/test/pytest/pyt/test_query_helpers.py
create mode 100755 src/tools/generate_pytest_libpq_errors.py
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index d80dd46c5fd..1440815b23a 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -840,7 +840,7 @@ float4:out:.*-.*-cygwin.*=float4-misrounded-input.out
</sect1>
<sect1 id="regress-tap">
- <title>TAP Tests</title>
+ <title>Perl TAP Tests</title>
<para>
Various tests, particularly the client program tests
@@ -929,6 +929,58 @@ PG_TEST_NOCLEAN=1 make -C src/bin/pg_dump check
</sect1>
+ <sect1 id="regress-pytest">
+ <title>Pytest Tests</title>
+
+ <para>
+ Tests in <filename>pyt</filename> directories use the Python
+ <application>pytest</application> framework. These tests provide a
+ convenient way to test libpq client functionality and scenarios requiring
+ multiple PostgreSQL server instances.
+ </para>
+
+ <para>
+ The pytest tests require <productname>PostgreSQL</productname> to be
+ configured with the option <option>--enable-pytest</option> (or
+ <option>-Dpytest=enabled</option> for Meson builds). You also need either
+ <application>pytest</application> or <application>uv</application>
+ installed on your system.
+ </para>
+
+ <para>
+ With Meson builds, you can run the pytest tests using:
+<programlisting>
+meson test --suite pytest
+</programlisting>
+ With autoconf-based builds, you can run them from the
+ <filename>src/test/pytest</filename> directory using:
+<programlisting>
+make check
+</programlisting>
+ </para>
+
+ <para>
+ You can also run specific test files directly using pytest:
+<programlisting>
+pytest src/test/pytest/pyt/test_libpq.py
+pytest -k "test_connstr"
+</programlisting>
+ </para>
+
+ <para>
+ Many operations in the test suites use a 180-second timeout, which on slow
+ hosts may lead to load-induced timeouts. Setting the environment variable
+ <varname>PG_TEST_TIMEOUT_DEFAULT</varname> to a higher number will change
+ the default to avoid this.
+ </para>
+
+ <para>
+ For more information on writing pytest tests, see the
+ <filename>src/test/pytest/README</filename> file.
+ </para>
+
+ </sect1>
+
<sect1 id="regress-coverage">
<title>Test Coverage Examination</title>
diff --git a/pyproject.toml b/pyproject.toml
index 60abb4d0655..4628d2274e0 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -19,3 +19,6 @@ minversion = "7.0"
# Common test code can be found here.
pythonpath = ["src/test/pytest"]
+
+# Load the shared fixtures plugin
+addopts = ["-p", "pypg.fixtures"]
diff --git a/src/backend/utils/errcodes.txt b/src/backend/utils/errcodes.txt
index c96aa7c49ef..40c7555047e 100644
--- a/src/backend/utils/errcodes.txt
+++ b/src/backend/utils/errcodes.txt
@@ -21,6 +21,11 @@
# doc/src/sgml/errcodes-table.sgml
# a SGML table of error codes for inclusion in the documentation
#
+# src/test/pytest/libpq/_generated_errors.py
+# Python exception classes for the pytest libpq wrapper
+# Note: This needs to be manually regenerated by running
+# src/tools/generate_pytest_libpq_errors.py
+#
# The format of this file is one error code per line, with the following
# whitespace-separated fields:
#
diff --git a/src/test/pytest/README b/src/test/pytest/README
index 1333ed77b7e..9dc50ca111f 100644
--- a/src/test/pytest/README
+++ b/src/test/pytest/README
@@ -1 +1,139 @@
-TODO
+src/test/pytest/README
+
+Pytest-based tests
+==================
+
+This directory contains infrastructure for Python-based tests using pytest,
+along with some core tests for the pytest infrastructure itself. The framework
+provides fixtures for managing PostgreSQL server instances and connecting to
+them via libpq.
+
+
+Running the tests
+=================
+
+NOTE: You must have given the --enable-pytest argument to configure (or
+-Dpytest=enabled for Meson builds). You also need to have either pytest or uv
+already installed.
+
+With Meson builds, you can run:
+ meson test --suite pytest
+
+With autoconf based builds, you can run:
+ make check
+or
+ make installcheck
+
+You can run specific test files and/or use pytest's -k option to select tests:
+ pytest src/test/pytest/pyt/test_libpq.py
+ pytest -k "test_connstr"
+
+
+Directory structure
+===================
+
+pypg/
+ Python library providing common functions and pytest fixtures that can be
+ used in tests.
+
+libpq/
+ A simple but user-friendly python wrapper around libpq
+
+pyt/
+ Tests for the pytest infrastructure itself
+
+pgtap.py
+ A pytest plugin to output results in TAP format
+
+
+Writing tests
+=============
+
+Tests use pytest fixtures to manage server instances and connections. The
+most commonly used fixtures are:
+
+pg
+ A PostgresServer instance configured for the current test. Use this for
+ creating test users/databases or modifying server configuration. Changes
+ are automatically rolled back after the test.
+
+conn
+ A connected PGconn instance to the test server. Automatically cleaned up
+ after the test.
+
+connect
+ A function to create additional connections with custom options.
+
+create_pg
+ A factory function to create additional PostgreSQL servers within a test.
+ Servers are automatically cleaned up at the end of the test. Useful for
+ testing scenarios that require multiple independent servers.
+
+create_pg_module
+ Like create_pg, but servers persist for the entire test module. Use this
+ when multiple tests in a module can share the same servers, which is
+ faster than creating new servers for each test.
+
+
+Example test:
+
+ def test_simple_query(conn):
+ result = conn.sql("SELECT 1 + 1")
+ assert result == 2
+
+ def test_with_user(pg):
+ users = pg.create_users("test")
+ with pg.reloading() as s:
+ s.hba.prepend(["local", "all", users["test"], "trust"])
+
+ conn = pg.connect(user=users["test"])
+ assert conn.sql("SELECT current_user") == users["test"]
+
+ def test_multiple_servers(create_pg):
+ node1 = create_pg("primary")
+ node2 = create_pg("secondary")
+
+ conn1 = node1.connect()
+ conn2 = node2.connect()
+
+ # Each server is independent
+ assert node1.port != node2.port
+
+
+Server configuration
+====================
+
+Tests can temporarily modify server configuration using context managers:
+
+ with pg.reloading() as s:
+ s.conf.set(log_connections="on")
+ s.hba.prepend("local all all trust")
+ # Server is reloaded here
+ # After the test finished the original configuration is restored and
+ # the server is reloaded again
+
+Use pg.restarting() instead if the configuration change requires a restart.
+
+
+Timeouts
+========
+
+Tests inherit the PG_TEST_TIMEOUT_DEFAULT environment variable (defaulting
+to 180 seconds). The remaining_timeout fixture provides a function that
+returns how much time remains for the current test.
+
+
+Environment variables
+=====================
+
+PG_TEST_TIMEOUT_DEFAULT
+ Per-test timeout in seconds (default: 180)
+
+PG_CONFIG
+ Path to pg_config (default: uses PATH)
+
+TESTDATADIR
+ Directory for test data (default: pytest temp directory)
+
+PG_TEST_EXTRA
+ Space-separated list of optional test categories to run (e.g., "ssl")
diff --git a/src/test/pytest/libpq/__init__.py b/src/test/pytest/libpq/__init__.py
new file mode 100644
index 00000000000..cb4d18b6206
--- /dev/null
+++ b/src/test/pytest/libpq/__init__.py
@@ -0,0 +1,36 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+libpq testing utilities - ctypes bindings and helpers for PostgreSQL's libpq library.
+
+This module provides Python wrappers around libpq for use in pytest tests.
+"""
+
+from . import errors
+from .errors import LibpqError, LibpqWarning
+from ._core import (
+ ConnectionStatus,
+ DiagField,
+ ExecStatus,
+ PGconn,
+ PGresult,
+ connect,
+ connstr,
+ load_libpq_handle,
+ register_type_info,
+)
+
+__all__ = [
+ "errors",
+ "LibpqError",
+ "LibpqWarning",
+ "ConnectionStatus",
+ "DiagField",
+ "ExecStatus",
+ "PGconn",
+ "PGresult",
+ "connect",
+ "connstr",
+ "load_libpq_handle",
+ "register_type_info",
+]
diff --git a/src/test/pytest/libpq/_core.py b/src/test/pytest/libpq/_core.py
new file mode 100644
index 00000000000..0d77996d572
--- /dev/null
+++ b/src/test/pytest/libpq/_core.py
@@ -0,0 +1,489 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Core libpq functionality - ctypes bindings and connection handling.
+"""
+
+import contextlib
+import ctypes
+import datetime
+import decimal
+import enum
+import json
+import platform
+import os
+import uuid
+from typing import Any, Callable, Dict, Optional
+
+from .errors import LibpqError, make_error
+
+
+# PG_DIAG field identifiers from postgres_ext.h
+class DiagField(enum.IntEnum):
+ SEVERITY = ord("S")
+ SEVERITY_NONLOCALIZED = ord("V")
+ SQLSTATE = ord("C")
+ MESSAGE_PRIMARY = ord("M")
+ MESSAGE_DETAIL = ord("D")
+ MESSAGE_HINT = ord("H")
+ STATEMENT_POSITION = ord("P")
+ INTERNAL_POSITION = ord("p")
+ INTERNAL_QUERY = ord("q")
+ CONTEXT = ord("W")
+ SCHEMA_NAME = ord("s")
+ TABLE_NAME = ord("t")
+ COLUMN_NAME = ord("c")
+ DATATYPE_NAME = ord("d")
+ CONSTRAINT_NAME = ord("n")
+ SOURCE_FILE = ord("F")
+ SOURCE_LINE = ord("L")
+ SOURCE_FUNCTION = ord("R")
+
+
+class ConnectionStatus(enum.IntEnum):
+ """PostgreSQL connection status codes from libpq."""
+
+ CONNECTION_OK = 0
+ CONNECTION_BAD = 1
+
+
+class ExecStatus(enum.IntEnum):
+ """PostgreSQL result status codes from PQresultStatus."""
+
+ PGRES_EMPTY_QUERY = 0
+ PGRES_COMMAND_OK = 1
+ PGRES_TUPLES_OK = 2
+ PGRES_COPY_OUT = 3
+ PGRES_COPY_IN = 4
+ PGRES_BAD_RESPONSE = 5
+ PGRES_NONFATAL_ERROR = 6
+ PGRES_FATAL_ERROR = 7
+ PGRES_COPY_BOTH = 8
+ PGRES_SINGLE_TUPLE = 9
+ PGRES_PIPELINE_SYNC = 10
+ PGRES_PIPELINE_ABORTED = 11
+
+
+class _PGconn(ctypes.Structure):
+ pass
+
+
+class _PGresult(ctypes.Structure):
+ pass
+
+
+_PGconn_p = ctypes.POINTER(_PGconn)
+_PGresult_p = ctypes.POINTER(_PGresult)
+
+
+def load_libpq_handle(libdir, bindir):
+ """
+ Loads a ctypes handle for libpq. Some common function prototypes are
+ initialized for general use.
+ """
+ system = platform.system()
+
+ if system in ("Linux", "FreeBSD", "NetBSD", "OpenBSD"):
+ name = "libpq.so.5"
+ elif system == "Darwin":
+ name = "libpq.5.dylib"
+ elif system == "Windows":
+ name = "libpq.dll"
+ else:
+ assert False, f"the libpq fixture must be updated for {system}"
+
+ if system == "Windows":
+ # On Windows, libpq.dll is confusingly in bindir, not libdir. And we
+ # need to add this directory the the search path.
+ libpq_path = os.path.join(bindir, name)
+ lib = ctypes.CDLL(libpq_path)
+ else:
+ libpq_path = os.path.join(libdir, name)
+ lib = ctypes.CDLL(libpq_path)
+
+ #
+ # Function Prototypes
+ #
+
+ lib.PQconnectdb.restype = _PGconn_p
+ lib.PQconnectdb.argtypes = [ctypes.c_char_p]
+
+ lib.PQstatus.restype = ctypes.c_int
+ lib.PQstatus.argtypes = [_PGconn_p]
+
+ lib.PQexec.restype = _PGresult_p
+ lib.PQexec.argtypes = [_PGconn_p, ctypes.c_char_p]
+
+ lib.PQresultStatus.restype = ctypes.c_int
+ lib.PQresultStatus.argtypes = [_PGresult_p]
+
+ lib.PQclear.restype = None
+ lib.PQclear.argtypes = [_PGresult_p]
+
+ lib.PQerrorMessage.restype = ctypes.c_char_p
+ lib.PQerrorMessage.argtypes = [_PGconn_p]
+
+ lib.PQfinish.restype = None
+ lib.PQfinish.argtypes = [_PGconn_p]
+
+ lib.PQresultErrorMessage.restype = ctypes.c_char_p
+ lib.PQresultErrorMessage.argtypes = [_PGresult_p]
+
+ lib.PQntuples.restype = ctypes.c_int
+ lib.PQntuples.argtypes = [_PGresult_p]
+
+ lib.PQnfields.restype = ctypes.c_int
+ lib.PQnfields.argtypes = [_PGresult_p]
+
+ lib.PQgetvalue.restype = ctypes.c_char_p
+ lib.PQgetvalue.argtypes = [_PGresult_p, ctypes.c_int, ctypes.c_int]
+
+ lib.PQgetisnull.restype = ctypes.c_int
+ lib.PQgetisnull.argtypes = [_PGresult_p, ctypes.c_int, ctypes.c_int]
+
+ lib.PQftype.restype = ctypes.c_uint
+ lib.PQftype.argtypes = [_PGresult_p, ctypes.c_int]
+
+ lib.PQresultErrorField.restype = ctypes.c_char_p
+ lib.PQresultErrorField.argtypes = [_PGresult_p, ctypes.c_int]
+
+ return lib
+
+
+# PostgreSQL type OIDs and conversion system
+# Type registry - maps OID to converter function
+_type_converters: Dict[int, Callable[[str], Any]] = {}
+_array_to_elem_map: Dict[int, int] = {}
+
+
+def register_type_info(
+ name: str, oid: int, array_oid: int, converter: Callable[[str], Any]
+):
+ """
+ Register a PostgreSQL type with its OID, array OID, and conversion function.
+
+ Usage:
+ register_type_info("bool", 16, 1000, lambda v: v == "t")
+ """
+ _type_converters[oid] = converter
+ if array_oid is not None:
+ _array_to_elem_map[array_oid] = oid
+
+
+def _parse_array(value: str, elem_oid: int):
+ """Parse PostgreSQL array syntax into nested Python lists."""
+ stack: list[list] = []
+ current_element: list[str] = []
+ in_quotes = False
+ was_quoted = False
+ pos = 0
+
+ while pos < len(value):
+ char = value[pos]
+
+ if in_quotes:
+ if char == "\\":
+ next_char = value[pos + 1]
+ if next_char not in '"\\':
+ raise NotImplementedError('Only \\" and \\\\ escapes are supported')
+ current_element.append(next_char)
+ pos += 2
+ continue
+ elif char == '"':
+ in_quotes = False
+ else:
+ current_element.append(char)
+ elif char == '"':
+ in_quotes = True
+ was_quoted = True
+ elif char == "{":
+ stack.append([])
+ elif char in ",}":
+ if current_element or was_quoted:
+ elem = "".join(current_element)
+ if not was_quoted and elem == "NULL":
+ stack[-1].append(None)
+ else:
+ stack[-1].append(_convert_pg_value(elem, elem_oid))
+ current_element = []
+ was_quoted = False
+ if char == "}":
+ completed = stack.pop()
+ if not stack:
+ return completed
+ stack[-1].append(completed)
+ elif char != " ":
+ current_element.append(char)
+ pos += 1
+
+ raise ValueError(f"Malformed array literal: {value}")
+
+
+# Register standard PostgreSQL types that we'll likely encounter in tests
+register_type_info("bool", 16, 1000, lambda v: v == "t")
+register_type_info("int2", 21, 1005, int)
+register_type_info("int4", 23, 1007, int)
+register_type_info("int8", 20, 1016, int)
+register_type_info("float4", 700, 1021, float)
+register_type_info("float8", 701, 1022, float)
+register_type_info("numeric", 1700, 1231, decimal.Decimal)
+register_type_info("text", 25, 1009, str)
+register_type_info("varchar", 1043, 1015, str)
+register_type_info("date", 1082, 1182, datetime.date.fromisoformat)
+register_type_info("time", 1083, 1183, datetime.time.fromisoformat)
+register_type_info("timestamp", 1114, 1115, datetime.datetime.fromisoformat)
+register_type_info("timestamptz", 1184, 1185, datetime.datetime.fromisoformat)
+register_type_info("uuid", 2950, 2951, uuid.UUID)
+register_type_info("json", 114, 199, json.loads)
+register_type_info("jsonb", 3802, 3807, json.loads)
+
+
+def _convert_pg_value(value: str, type_oid: int) -> Any:
+ """
+ Convert PostgreSQL string value to appropriate Python type based on OID.
+ Uses the registered type converters from register_type_info().
+ """
+ # Check if it's an array type
+ if type_oid in _array_to_elem_map:
+ elem_oid = _array_to_elem_map[type_oid]
+ return _parse_array(value, elem_oid)
+
+ # Use registered converter if available
+ converter = _type_converters.get(type_oid)
+ if converter:
+ return converter(value)
+
+ # Unknown types - return as string
+ return value
+
+
+def simplify_query_results(results) -> Any:
+ """
+ Simplify the results of a query so that the caller doesn't have to unpack
+ lists and tuples of length 1.
+ """
+ if len(results) == 1:
+ row = results[0]
+ if len(row) == 1:
+ # If there's only a single cell, just return the value
+ return row[0]
+ # If there's only a single row, just return that row
+ return row
+
+ if len(results) != 0 and len(results[0]) == 1:
+ # If there's only a single column, return an array of values
+ return [row[0] for row in results]
+
+ # if there are multiple rows and columns, return the results as is
+ return results
+
+
+class PGresult(contextlib.AbstractContextManager):
+ """Wraps a raw _PGresult_p with a more friendly interface."""
+
+ def __init__(self, lib: ctypes.CDLL, res: _PGresult_p):
+ self._lib = lib
+ self._res = res
+
+ def __exit__(self, *exc):
+ self._lib.PQclear(self._res)
+ self._res = None
+
+ def status(self) -> ExecStatus:
+ return ExecStatus(self._lib.PQresultStatus(self._res))
+
+ def error_message(self):
+ """Returns the error message associated with this result."""
+ msg = self._lib.PQresultErrorMessage(self._res)
+ return msg.decode() if msg else ""
+
+ def _get_error_field(self, field: DiagField) -> Optional[str]:
+ """Get an error field from the result using PQresultErrorField."""
+ val = self._lib.PQresultErrorField(self._res, int(field))
+ return val.decode() if val else None
+
+ def raise_error(self) -> None:
+ """
+ Raises an appropriate LibpqError subclass based on the error fields.
+ Extracts SQLSTATE and other diagnostic information from the result.
+ """
+ if not self._res:
+ raise LibpqError("query failed: out of memory or connection lost")
+
+ sqlstate = self._get_error_field(DiagField.SQLSTATE)
+ primary = self._get_error_field(DiagField.MESSAGE_PRIMARY)
+ detail = self._get_error_field(DiagField.MESSAGE_DETAIL)
+ hint = self._get_error_field(DiagField.MESSAGE_HINT)
+ severity = self._get_error_field(DiagField.SEVERITY)
+ schema_name = self._get_error_field(DiagField.SCHEMA_NAME)
+ table_name = self._get_error_field(DiagField.TABLE_NAME)
+ column_name = self._get_error_field(DiagField.COLUMN_NAME)
+ datatype_name = self._get_error_field(DiagField.DATATYPE_NAME)
+ constraint_name = self._get_error_field(DiagField.CONSTRAINT_NAME)
+ context = self._get_error_field(DiagField.CONTEXT)
+
+ position_str = self._get_error_field(DiagField.STATEMENT_POSITION)
+ position = int(position_str) if position_str else None
+
+ raise make_error(
+ primary or self.error_message(),
+ sqlstate=sqlstate,
+ severity=severity,
+ primary=primary,
+ detail=detail,
+ hint=hint,
+ schema_name=schema_name,
+ table_name=table_name,
+ column_name=column_name,
+ datatype_name=datatype_name,
+ constraint_name=constraint_name,
+ position=position,
+ context=context,
+ )
+
+ def fetch_all(self):
+ """
+ Fetch all rows and convert to Python types.
+ Returns a list of tuples, with values converted based on their PostgreSQL type.
+ """
+ nrows = self._lib.PQntuples(self._res)
+ ncols = self._lib.PQnfields(self._res)
+
+ # Get type OIDs for each column
+ type_oids = [self._lib.PQftype(self._res, col) for col in range(ncols)]
+
+ results = []
+ for row in range(nrows):
+ row_data = []
+ for col in range(ncols):
+ if self._lib.PQgetisnull(self._res, row, col):
+ row_data.append(None)
+ else:
+ value = self._lib.PQgetvalue(self._res, row, col).decode()
+ row_data.append(_convert_pg_value(value, type_oids[col]))
+ results.append(tuple(row_data))
+
+ return results
+
+
+class PGconn(contextlib.AbstractContextManager):
+ """
+ Wraps a raw _PGconn_p with a more friendly interface. This is just a
+ stub; it's expected to grow.
+ """
+
+ def __init__(
+ self,
+ lib: ctypes.CDLL,
+ handle: _PGconn_p,
+ stack: contextlib.ExitStack,
+ ):
+ self._lib = lib
+ self._handle = handle
+ self._stack = stack
+
+ def __exit__(self, *exc):
+ self._lib.PQfinish(self._handle)
+ self._handle = None
+
+ def exec(self, query: str):
+ """
+ Executes a query via PQexec() and returns a PGresult.
+ """
+ res = self._lib.PQexec(self._handle, query.encode())
+ return self._stack.enter_context(PGresult(self._lib, res))
+
+ def sql(self, query: str):
+ """
+ Executes a query and raises an exception if it fails.
+ Returns the query results with automatic type conversion and simplification.
+ For commands that don't return data (INSERT, UPDATE, etc.), returns None.
+
+ Examples:
+ - SELECT 1 -> 1
+ - SELECT 1, 2 -> (1, 2)
+ - SELECT * FROM generate_series(1, 3) -> [1, 2, 3]
+ - SELECT * FROM (VALUES (1, 'a'), (2, 'b')) t -> [(1, 'a'), (2, 'b')]
+ - CREATE TABLE ... -> None
+ - INSERT INTO ... -> None
+ """
+ res = self.exec(query)
+ status = res.status()
+
+ if status == ExecStatus.PGRES_FATAL_ERROR:
+ res.raise_error()
+ elif status == ExecStatus.PGRES_COMMAND_OK:
+ return None
+ elif status == ExecStatus.PGRES_TUPLES_OK:
+ results = res.fetch_all()
+ return simplify_query_results(results)
+ else:
+ res.raise_error()
+
+
+def connstr(opts: Dict[str, Any]) -> str:
+ """
+ Flattens the provided options into a libpq connection string. Values
+ are converted to str and quoted/escaped as necessary.
+ """
+ settings = []
+
+ for k, v in opts.items():
+ v = str(v)
+ if not v:
+ v = "''"
+ else:
+ v = v.replace("\\", "\\\\")
+ v = v.replace("'", "\\'")
+
+ if " " in v:
+ v = f"'{v}'"
+
+ settings.append(f"{k}={v}")
+
+ return " ".join(settings)
+
+
+def connect(
+ libpq_handle: ctypes.CDLL,
+ stack: contextlib.ExitStack,
+ remaining_timeout_fn: Callable[[], float],
+ **opts,
+) -> PGconn:
+ """
+ Connects to a server, using the given connection options, and
+ returns a PGconn object wrapping the connection handle. A
+ failure will raise LibpqError.
+
+ Connections honor PG_TEST_TIMEOUT_DEFAULT unless connect_timeout is
+ explicitly overridden in opts.
+
+ Args:
+ libpq_handle: ctypes.CDLL handle to libpq library
+ stack: ExitStack for managing connection cleanup
+ remaining_timeout_fn: Function that returns remaining timeout in seconds
+ **opts: Connection options (host, port, dbname, etc.)
+
+ Returns:
+ PGconn: Connected database connection
+
+ Raises:
+ LibpqError: If connection fails
+ """
+
+ if "connect_timeout" not in opts:
+ t = int(remaining_timeout_fn())
+ opts["connect_timeout"] = max(t, 1)
+
+ conn_p = libpq_handle.PQconnectdb(connstr(opts).encode())
+
+ # Check connection status before adding to stack
+ if libpq_handle.PQstatus(conn_p) != ConnectionStatus.CONNECTION_OK:
+ error_msg = libpq_handle.PQerrorMessage(conn_p).decode()
+ # Manually close the failed connection
+ libpq_handle.PQfinish(conn_p)
+ raise LibpqError(error_msg)
+
+ # Connection succeeded - add to stack for cleanup
+ conn = stack.enter_context(PGconn(libpq_handle, conn_p, stack=stack))
+ return conn
diff --git a/src/test/pytest/libpq/_error_base.py b/src/test/pytest/libpq/_error_base.py
new file mode 100644
index 00000000000..5c70c077193
--- /dev/null
+++ b/src/test/pytest/libpq/_error_base.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Base exception classes for libpq errors and warnings.
+"""
+
+from typing import Optional
+
+
+class LibpqExceptionMixin:
+ """Mixin providing PostgreSQL error field attributes."""
+
+ sqlstate: Optional[str]
+ severity: Optional[str]
+ primary: Optional[str]
+ detail: Optional[str]
+ hint: Optional[str]
+ schema_name: Optional[str]
+ table_name: Optional[str]
+ column_name: Optional[str]
+ datatype_name: Optional[str]
+ constraint_name: Optional[str]
+ position: Optional[int]
+ context: Optional[str]
+
+ def __init__(
+ self,
+ message: str,
+ *,
+ sqlstate: Optional[str] = None,
+ severity: Optional[str] = None,
+ primary: Optional[str] = None,
+ detail: Optional[str] = None,
+ hint: Optional[str] = None,
+ schema_name: Optional[str] = None,
+ table_name: Optional[str] = None,
+ column_name: Optional[str] = None,
+ datatype_name: Optional[str] = None,
+ constraint_name: Optional[str] = None,
+ position: Optional[int] = None,
+ context: Optional[str] = None,
+ ):
+ super().__init__(message)
+ self.sqlstate = sqlstate
+ self.severity = severity
+ self.primary = primary
+ self.detail = detail
+ self.hint = hint
+ self.schema_name = schema_name
+ self.table_name = table_name
+ self.column_name = column_name
+ self.datatype_name = datatype_name
+ self.constraint_name = constraint_name
+ self.position = position
+ self.context = context
+
+ @property
+ def sqlstate_class(self) -> Optional[str]:
+ """Returns the 2-character SQLSTATE class."""
+ if self.sqlstate and len(self.sqlstate) >= 2:
+ return self.sqlstate[:2]
+ return None
+
+
+class LibpqError(LibpqExceptionMixin, RuntimeError):
+ """Base exception for libpq errors."""
+
+ pass
+
+
+class LibpqWarning(LibpqExceptionMixin, UserWarning):
+ """Base exception for libpq warnings."""
+
+ pass
diff --git a/src/test/pytest/libpq/_generated_errors.py b/src/test/pytest/libpq/_generated_errors.py
new file mode 100644
index 00000000000..f50f3143580
--- /dev/null
+++ b/src/test/pytest/libpq/_generated_errors.py
@@ -0,0 +1,2116 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+# This file is generated by src/tools/generate_pytest_libpq_errors.py - do not edit directly.
+
+"""
+Generated PostgreSQL error classes mapped from SQLSTATE codes.
+"""
+
+from typing import Dict
+
+from ._error_base import LibpqError, LibpqWarning
+
+
+class SuccessfulCompletion(LibpqError):
+ """SQLSTATE 00000 - successful completion."""
+
+ pass
+
+
+class Warning(LibpqWarning):
+ """SQLSTATE 01000 - warning."""
+
+ pass
+
+
+class DynamicResultSetsReturnedWarning(Warning):
+ """SQLSTATE 0100C - dynamic result sets returned."""
+
+ pass
+
+
+class ImplicitZeroBitPaddingWarning(Warning):
+ """SQLSTATE 01008 - implicit zero bit padding."""
+
+ pass
+
+
+class NullValueEliminatedInSetFunctionWarning(Warning):
+ """SQLSTATE 01003 - null value eliminated in set function."""
+
+ pass
+
+
+class PrivilegeNotGrantedWarning(Warning):
+ """SQLSTATE 01007 - privilege not granted."""
+
+ pass
+
+
+class PrivilegeNotRevokedWarning(Warning):
+ """SQLSTATE 01006 - privilege not revoked."""
+
+ pass
+
+
+class StringDataRightTruncationWarning(Warning):
+ """SQLSTATE 01004 - string data right truncation."""
+
+ pass
+
+
+class DeprecatedFeatureWarning(Warning):
+ """SQLSTATE 01P01 - deprecated feature."""
+
+ pass
+
+
+class NoData(LibpqError):
+ """SQLSTATE 02000 - no data."""
+
+ pass
+
+
+class NoAdditionalDynamicResultSetsReturned(NoData):
+ """SQLSTATE 02001 - no additional dynamic result sets returned."""
+
+ pass
+
+
+class SQLStatementNotYetComplete(LibpqError):
+ """SQLSTATE 03000 - sql statement not yet complete."""
+
+ pass
+
+
+class ConnectionException(LibpqError):
+ """SQLSTATE 08000 - connection exception."""
+
+ pass
+
+
+class ConnectionDoesNotExist(ConnectionException):
+ """SQLSTATE 08003 - connection does not exist."""
+
+ pass
+
+
+class ConnectionFailure(ConnectionException):
+ """SQLSTATE 08006 - connection failure."""
+
+ pass
+
+
+class SQLClientUnableToEstablishSQLConnection(ConnectionException):
+ """SQLSTATE 08001 - sqlclient unable to establish sqlconnection."""
+
+ pass
+
+
+class SQLServerRejectedEstablishmentOfSQLConnection(ConnectionException):
+ """SQLSTATE 08004 - sqlserver rejected establishment of sqlconnection."""
+
+ pass
+
+
+class TransactionResolutionUnknown(ConnectionException):
+ """SQLSTATE 08007 - transaction resolution unknown."""
+
+ pass
+
+
+class ProtocolViolation(ConnectionException):
+ """SQLSTATE 08P01 - protocol violation."""
+
+ pass
+
+
+class TriggeredActionException(LibpqError):
+ """SQLSTATE 09000 - triggered action exception."""
+
+ pass
+
+
+class FeatureNotSupported(LibpqError):
+ """SQLSTATE 0A000 - feature not supported."""
+
+ pass
+
+
+class InvalidTransactionInitiation(LibpqError):
+ """SQLSTATE 0B000 - invalid transaction initiation."""
+
+ pass
+
+
+class LocatorException(LibpqError):
+ """SQLSTATE 0F000 - locator exception."""
+
+ pass
+
+
+class InvalidLocatorSpecification(LocatorException):
+ """SQLSTATE 0F001 - invalid locator specification."""
+
+ pass
+
+
+class InvalidGrantor(LibpqError):
+ """SQLSTATE 0L000 - invalid grantor."""
+
+ pass
+
+
+class InvalidGrantOperation(InvalidGrantor):
+ """SQLSTATE 0LP01 - invalid grant operation."""
+
+ pass
+
+
+class InvalidRoleSpecification(LibpqError):
+ """SQLSTATE 0P000 - invalid role specification."""
+
+ pass
+
+
+class DiagnosticsException(LibpqError):
+ """SQLSTATE 0Z000 - diagnostics exception."""
+
+ pass
+
+
+class StackedDiagnosticsAccessedWithoutActiveHandler(DiagnosticsException):
+ """SQLSTATE 0Z002 - stacked diagnostics accessed without active handler."""
+
+ pass
+
+
+class InvalidArgumentForXquery(LibpqError):
+ """SQLSTATE 10608 - invalid argument for xquery."""
+
+ pass
+
+
+class CaseNotFound(LibpqError):
+ """SQLSTATE 20000 - case not found."""
+
+ pass
+
+
+class CardinalityViolation(LibpqError):
+ """SQLSTATE 21000 - cardinality violation."""
+
+ pass
+
+
+class DataException(LibpqError):
+ """SQLSTATE 22000 - data exception."""
+
+ pass
+
+
+class ArraySubscriptError(DataException):
+ """SQLSTATE 2202E - array subscript error."""
+
+ pass
+
+
+class CharacterNotInRepertoire(DataException):
+ """SQLSTATE 22021 - character not in repertoire."""
+
+ pass
+
+
+class DatetimeFieldOverflow(DataException):
+ """SQLSTATE 22008 - datetime field overflow."""
+
+ pass
+
+
+class DivisionByZero(DataException):
+ """SQLSTATE 22012 - division by zero."""
+
+ pass
+
+
+class ErrorInAssignment(DataException):
+ """SQLSTATE 22005 - error in assignment."""
+
+ pass
+
+
+class EscapeCharacterConflict(DataException):
+ """SQLSTATE 2200B - escape character conflict."""
+
+ pass
+
+
+class IndicatorOverflow(DataException):
+ """SQLSTATE 22022 - indicator overflow."""
+
+ pass
+
+
+class IntervalFieldOverflow(DataException):
+ """SQLSTATE 22015 - interval field overflow."""
+
+ pass
+
+
+class InvalidArgumentForLogarithm(DataException):
+ """SQLSTATE 2201E - invalid argument for logarithm."""
+
+ pass
+
+
+class InvalidArgumentForNtileFunction(DataException):
+ """SQLSTATE 22014 - invalid argument for ntile function."""
+
+ pass
+
+
+class InvalidArgumentForNthValueFunction(DataException):
+ """SQLSTATE 22016 - invalid argument for nth value function."""
+
+ pass
+
+
+class InvalidArgumentForPowerFunction(DataException):
+ """SQLSTATE 2201F - invalid argument for power function."""
+
+ pass
+
+
+class InvalidArgumentForWidthBucketFunction(DataException):
+ """SQLSTATE 2201G - invalid argument for width bucket function."""
+
+ pass
+
+
+class InvalidCharacterValueForCast(DataException):
+ """SQLSTATE 22018 - invalid character value for cast."""
+
+ pass
+
+
+class InvalidDatetimeFormat(DataException):
+ """SQLSTATE 22007 - invalid datetime format."""
+
+ pass
+
+
+class InvalidEscapeCharacter(DataException):
+ """SQLSTATE 22019 - invalid escape character."""
+
+ pass
+
+
+class InvalidEscapeOctet(DataException):
+ """SQLSTATE 2200D - invalid escape octet."""
+
+ pass
+
+
+class InvalidEscapeSequence(DataException):
+ """SQLSTATE 22025 - invalid escape sequence."""
+
+ pass
+
+
+class NonstandardUseOfEscapeCharacter(DataException):
+ """SQLSTATE 22P06 - nonstandard use of escape character."""
+
+ pass
+
+
+class InvalidIndicatorParameterValue(DataException):
+ """SQLSTATE 22010 - invalid indicator parameter value."""
+
+ pass
+
+
+class InvalidParameterValue(DataException):
+ """SQLSTATE 22023 - invalid parameter value."""
+
+ pass
+
+
+class InvalidPrecedingOrFollowingSize(DataException):
+ """SQLSTATE 22013 - invalid preceding or following size."""
+
+ pass
+
+
+class InvalidRegularExpression(DataException):
+ """SQLSTATE 2201B - invalid regular expression."""
+
+ pass
+
+
+class InvalidRowCountInLimitClause(DataException):
+ """SQLSTATE 2201W - invalid row count in limit clause."""
+
+ pass
+
+
+class InvalidRowCountInResultOffsetClause(DataException):
+ """SQLSTATE 2201X - invalid row count in result offset clause."""
+
+ pass
+
+
+class InvalidTablesampleArgument(DataException):
+ """SQLSTATE 2202H - invalid tablesample argument."""
+
+ pass
+
+
+class InvalidTablesampleRepeat(DataException):
+ """SQLSTATE 2202G - invalid tablesample repeat."""
+
+ pass
+
+
+class InvalidTimeZoneDisplacementValue(DataException):
+ """SQLSTATE 22009 - invalid time zone displacement value."""
+
+ pass
+
+
+class InvalidUseOfEscapeCharacter(DataException):
+ """SQLSTATE 2200C - invalid use of escape character."""
+
+ pass
+
+
+class MostSpecificTypeMismatch(DataException):
+ """SQLSTATE 2200G - most specific type mismatch."""
+
+ pass
+
+
+class NullValueNotAllowed(DataException):
+ """SQLSTATE 22004 - null value not allowed."""
+
+ pass
+
+
+class NullValueNoIndicatorParameter(DataException):
+ """SQLSTATE 22002 - null value no indicator parameter."""
+
+ pass
+
+
+class NumericValueOutOfRange(DataException):
+ """SQLSTATE 22003 - numeric value out of range."""
+
+ pass
+
+
+class SequenceGeneratorLimitExceeded(DataException):
+ """SQLSTATE 2200H - sequence generator limit exceeded."""
+
+ pass
+
+
+class StringDataLengthMismatch(DataException):
+ """SQLSTATE 22026 - string data length mismatch."""
+
+ pass
+
+
+class StringDataRightTruncation(DataException):
+ """SQLSTATE 22001 - string data right truncation."""
+
+ pass
+
+
+class SubstringError(DataException):
+ """SQLSTATE 22011 - substring error."""
+
+ pass
+
+
+class TrimError(DataException):
+ """SQLSTATE 22027 - trim error."""
+
+ pass
+
+
+class UnterminatedCString(DataException):
+ """SQLSTATE 22024 - unterminated c string."""
+
+ pass
+
+
+class ZeroLengthCharacterString(DataException):
+ """SQLSTATE 2200F - zero length character string."""
+
+ pass
+
+
+class FloatingPointException(DataException):
+ """SQLSTATE 22P01 - floating point exception."""
+
+ pass
+
+
+class InvalidTextRepresentation(DataException):
+ """SQLSTATE 22P02 - invalid text representation."""
+
+ pass
+
+
+class InvalidBinaryRepresentation(DataException):
+ """SQLSTATE 22P03 - invalid binary representation."""
+
+ pass
+
+
+class BadCopyFileFormat(DataException):
+ """SQLSTATE 22P04 - bad copy file format."""
+
+ pass
+
+
+class UntranslatableCharacter(DataException):
+ """SQLSTATE 22P05 - untranslatable character."""
+
+ pass
+
+
+class NotAnXmlDocument(DataException):
+ """SQLSTATE 2200L - not an xml document."""
+
+ pass
+
+
+class InvalidXmlDocument(DataException):
+ """SQLSTATE 2200M - invalid xml document."""
+
+ pass
+
+
+class InvalidXmlContent(DataException):
+ """SQLSTATE 2200N - invalid xml content."""
+
+ pass
+
+
+class InvalidXmlComment(DataException):
+ """SQLSTATE 2200S - invalid xml comment."""
+
+ pass
+
+
+class InvalidXmlProcessingInstruction(DataException):
+ """SQLSTATE 2200T - invalid xml processing instruction."""
+
+ pass
+
+
+class DuplicateJsonObjectKeyValue(DataException):
+ """SQLSTATE 22030 - duplicate json object key value."""
+
+ pass
+
+
+class InvalidArgumentForSQLJsonDatetimeFunction(DataException):
+ """SQLSTATE 22031 - invalid argument for sql json datetime function."""
+
+ pass
+
+
+class InvalidJsonText(DataException):
+ """SQLSTATE 22032 - invalid json text."""
+
+ pass
+
+
+class InvalidSQLJsonSubscript(DataException):
+ """SQLSTATE 22033 - invalid sql json subscript."""
+
+ pass
+
+
+class MoreThanOneSQLJsonItem(DataException):
+ """SQLSTATE 22034 - more than one sql json item."""
+
+ pass
+
+
+class NoSQLJsonItem(DataException):
+ """SQLSTATE 22035 - no sql json item."""
+
+ pass
+
+
+class NonNumericSQLJsonItem(DataException):
+ """SQLSTATE 22036 - non numeric sql json item."""
+
+ pass
+
+
+class NonUniqueKeysInAJsonObject(DataException):
+ """SQLSTATE 22037 - non unique keys in a json object."""
+
+ pass
+
+
+class SingletonSQLJsonItemRequired(DataException):
+ """SQLSTATE 22038 - singleton sql json item required."""
+
+ pass
+
+
+class SQLJsonArrayNotFound(DataException):
+ """SQLSTATE 22039 - sql json array not found."""
+
+ pass
+
+
+class SQLJsonMemberNotFound(DataException):
+ """SQLSTATE 2203A - sql json member not found."""
+
+ pass
+
+
+class SQLJsonNumberNotFound(DataException):
+ """SQLSTATE 2203B - sql json number not found."""
+
+ pass
+
+
+class SQLJsonObjectNotFound(DataException):
+ """SQLSTATE 2203C - sql json object not found."""
+
+ pass
+
+
+class TooManyJsonArrayElements(DataException):
+ """SQLSTATE 2203D - too many json array elements."""
+
+ pass
+
+
+class TooManyJsonObjectMembers(DataException):
+ """SQLSTATE 2203E - too many json object members."""
+
+ pass
+
+
+class SQLJsonScalarRequired(DataException):
+ """SQLSTATE 2203F - sql json scalar required."""
+
+ pass
+
+
+class SQLJsonItemCannotBeCastToTargetType(DataException):
+ """SQLSTATE 2203G - sql json item cannot be cast to target type."""
+
+ pass
+
+
+class IntegrityConstraintViolation(LibpqError):
+ """SQLSTATE 23000 - integrity constraint violation."""
+
+ pass
+
+
+class RestrictViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23001 - restrict violation."""
+
+ pass
+
+
+class NotNullViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23502 - not null violation."""
+
+ pass
+
+
+class ForeignKeyViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23503 - foreign key violation."""
+
+ pass
+
+
+class UniqueViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23505 - unique violation."""
+
+ pass
+
+
+class CheckViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23514 - check violation."""
+
+ pass
+
+
+class ExclusionViolation(IntegrityConstraintViolation):
+ """SQLSTATE 23P01 - exclusion violation."""
+
+ pass
+
+
+class InvalidCursorState(LibpqError):
+ """SQLSTATE 24000 - invalid cursor state."""
+
+ pass
+
+
+class InvalidTransactionState(LibpqError):
+ """SQLSTATE 25000 - invalid transaction state."""
+
+ pass
+
+
+class ActiveSQLTransaction(InvalidTransactionState):
+ """SQLSTATE 25001 - active sql transaction."""
+
+ pass
+
+
+class BranchTransactionAlreadyActive(InvalidTransactionState):
+ """SQLSTATE 25002 - branch transaction already active."""
+
+ pass
+
+
+class HeldCursorRequiresSameIsolationLevel(InvalidTransactionState):
+ """SQLSTATE 25008 - held cursor requires same isolation level."""
+
+ pass
+
+
+class InappropriateAccessModeForBranchTransaction(InvalidTransactionState):
+ """SQLSTATE 25003 - inappropriate access mode for branch transaction."""
+
+ pass
+
+
+class InappropriateIsolationLevelForBranchTransaction(InvalidTransactionState):
+ """SQLSTATE 25004 - inappropriate isolation level for branch transaction."""
+
+ pass
+
+
+class NoActiveSQLTransactionForBranchTransaction(InvalidTransactionState):
+ """SQLSTATE 25005 - no active sql transaction for branch transaction."""
+
+ pass
+
+
+class ReadOnlySQLTransaction(InvalidTransactionState):
+ """SQLSTATE 25006 - read only sql transaction."""
+
+ pass
+
+
+class SchemaAndDataStatementMixingNotSupported(InvalidTransactionState):
+ """SQLSTATE 25007 - schema and data statement mixing not supported."""
+
+ pass
+
+
+class NoActiveSQLTransaction(InvalidTransactionState):
+ """SQLSTATE 25P01 - no active sql transaction."""
+
+ pass
+
+
+class InFailedSQLTransaction(InvalidTransactionState):
+ """SQLSTATE 25P02 - in failed sql transaction."""
+
+ pass
+
+
+class IdleInTransactionSessionTimeout(InvalidTransactionState):
+ """SQLSTATE 25P03 - idle in transaction session timeout."""
+
+ pass
+
+
+class TransactionTimeout(InvalidTransactionState):
+ """SQLSTATE 25P04 - transaction timeout."""
+
+ pass
+
+
+class InvalidSQLStatementName(LibpqError):
+ """SQLSTATE 26000 - invalid sql statement name."""
+
+ pass
+
+
+class TriggeredDataChangeViolation(LibpqError):
+ """SQLSTATE 27000 - triggered data change violation."""
+
+ pass
+
+
+class InvalidAuthorizationSpecification(LibpqError):
+ """SQLSTATE 28000 - invalid authorization specification."""
+
+ pass
+
+
+class InvalidPassword(InvalidAuthorizationSpecification):
+ """SQLSTATE 28P01 - invalid password."""
+
+ pass
+
+
+class DependentPrivilegeDescriptorsStillExist(LibpqError):
+ """SQLSTATE 2B000 - dependent privilege descriptors still exist."""
+
+ pass
+
+
+class DependentObjectsStillExist(DependentPrivilegeDescriptorsStillExist):
+ """SQLSTATE 2BP01 - dependent objects still exist."""
+
+ pass
+
+
+class InvalidTransactionTermination(LibpqError):
+ """SQLSTATE 2D000 - invalid transaction termination."""
+
+ pass
+
+
+class SQLRoutineException(LibpqError):
+ """SQLSTATE 2F000 - sql routine exception."""
+
+ pass
+
+
+class FunctionExecutedNoReturnStatement(SQLRoutineException):
+ """SQLSTATE 2F005 - function executed no return statement."""
+
+ pass
+
+
+class SREModifyingSQLDataNotPermitted(SQLRoutineException):
+ """SQLSTATE 2F002 - modifying sql data not permitted."""
+
+ pass
+
+
+class SREProhibitedSQLStatementAttempted(SQLRoutineException):
+ """SQLSTATE 2F003 - prohibited sql statement attempted."""
+
+ pass
+
+
+class SREReadingSQLDataNotPermitted(SQLRoutineException):
+ """SQLSTATE 2F004 - reading sql data not permitted."""
+
+ pass
+
+
+class InvalidCursorName(LibpqError):
+ """SQLSTATE 34000 - invalid cursor name."""
+
+ pass
+
+
+class ExternalRoutineException(LibpqError):
+ """SQLSTATE 38000 - external routine exception."""
+
+ pass
+
+
+class ContainingSQLNotPermitted(ExternalRoutineException):
+ """SQLSTATE 38001 - containing sql not permitted."""
+
+ pass
+
+
+class EREModifyingSQLDataNotPermitted(ExternalRoutineException):
+ """SQLSTATE 38002 - modifying sql data not permitted."""
+
+ pass
+
+
+class EREProhibitedSQLStatementAttempted(ExternalRoutineException):
+ """SQLSTATE 38003 - prohibited sql statement attempted."""
+
+ pass
+
+
+class EREReadingSQLDataNotPermitted(ExternalRoutineException):
+ """SQLSTATE 38004 - reading sql data not permitted."""
+
+ pass
+
+
+class ExternalRoutineInvocationException(LibpqError):
+ """SQLSTATE 39000 - external routine invocation exception."""
+
+ pass
+
+
+class InvalidSqlstateReturned(ExternalRoutineInvocationException):
+ """SQLSTATE 39001 - invalid sqlstate returned."""
+
+ pass
+
+
+class ERIENullValueNotAllowed(ExternalRoutineInvocationException):
+ """SQLSTATE 39004 - null value not allowed."""
+
+ pass
+
+
+class TriggerProtocolViolated(ExternalRoutineInvocationException):
+ """SQLSTATE 39P01 - trigger protocol violated."""
+
+ pass
+
+
+class SrfProtocolViolated(ExternalRoutineInvocationException):
+ """SQLSTATE 39P02 - srf protocol violated."""
+
+ pass
+
+
+class EventTriggerProtocolViolated(ExternalRoutineInvocationException):
+ """SQLSTATE 39P03 - event trigger protocol violated."""
+
+ pass
+
+
+class SavepointException(LibpqError):
+ """SQLSTATE 3B000 - savepoint exception."""
+
+ pass
+
+
+class InvalidSavepointSpecification(SavepointException):
+ """SQLSTATE 3B001 - invalid savepoint specification."""
+
+ pass
+
+
+class InvalidCatalogName(LibpqError):
+ """SQLSTATE 3D000 - invalid catalog name."""
+
+ pass
+
+
+class InvalidSchemaName(LibpqError):
+ """SQLSTATE 3F000 - invalid schema name."""
+
+ pass
+
+
+class TransactionRollback(LibpqError):
+ """SQLSTATE 40000 - transaction rollback."""
+
+ pass
+
+
+class TransactionIntegrityConstraintViolation(TransactionRollback):
+ """SQLSTATE 40002 - transaction integrity constraint violation."""
+
+ pass
+
+
+class SerializationFailure(TransactionRollback):
+ """SQLSTATE 40001 - serialization failure."""
+
+ pass
+
+
+class StatementCompletionUnknown(TransactionRollback):
+ """SQLSTATE 40003 - statement completion unknown."""
+
+ pass
+
+
+class DeadlockDetected(TransactionRollback):
+ """SQLSTATE 40P01 - deadlock detected."""
+
+ pass
+
+
+class SyntaxErrorOrAccessRuleViolation(LibpqError):
+ """SQLSTATE 42000 - syntax error or access rule violation."""
+
+ pass
+
+
+class SyntaxError(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42601 - syntax error."""
+
+ pass
+
+
+class InsufficientPrivilege(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42501 - insufficient privilege."""
+
+ pass
+
+
+class CannotCoerce(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42846 - cannot coerce."""
+
+ pass
+
+
+class GroupingError(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42803 - grouping error."""
+
+ pass
+
+
+class WindowingError(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P20 - windowing error."""
+
+ pass
+
+
+class InvalidRecursion(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P19 - invalid recursion."""
+
+ pass
+
+
+class InvalidForeignKey(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42830 - invalid foreign key."""
+
+ pass
+
+
+class InvalidName(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42602 - invalid name."""
+
+ pass
+
+
+class NameTooLong(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42622 - name too long."""
+
+ pass
+
+
+class ReservedName(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42939 - reserved name."""
+
+ pass
+
+
+class DatatypeMismatch(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42804 - datatype mismatch."""
+
+ pass
+
+
+class IndeterminateDatatype(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P18 - indeterminate datatype."""
+
+ pass
+
+
+class CollationMismatch(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P21 - collation mismatch."""
+
+ pass
+
+
+class IndeterminateCollation(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P22 - indeterminate collation."""
+
+ pass
+
+
+class WrongObjectType(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42809 - wrong object type."""
+
+ pass
+
+
+class GeneratedAlways(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 428C9 - generated always."""
+
+ pass
+
+
+class UndefinedColumn(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42703 - undefined column."""
+
+ pass
+
+
+class UndefinedFunction(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42883 - undefined function."""
+
+ pass
+
+
+class UndefinedTable(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P01 - undefined table."""
+
+ pass
+
+
+class UndefinedParameter(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P02 - undefined parameter."""
+
+ pass
+
+
+class UndefinedObject(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42704 - undefined object."""
+
+ pass
+
+
+class DuplicateColumn(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42701 - duplicate column."""
+
+ pass
+
+
+class DuplicateCursor(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P03 - duplicate cursor."""
+
+ pass
+
+
+class DuplicateDatabase(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P04 - duplicate database."""
+
+ pass
+
+
+class DuplicateFunction(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42723 - duplicate function."""
+
+ pass
+
+
+class DuplicatePreparedStatement(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P05 - duplicate prepared statement."""
+
+ pass
+
+
+class DuplicateSchema(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P06 - duplicate schema."""
+
+ pass
+
+
+class DuplicateTable(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P07 - duplicate table."""
+
+ pass
+
+
+class DuplicateAlias(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42712 - duplicate alias."""
+
+ pass
+
+
+class DuplicateObject(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42710 - duplicate object."""
+
+ pass
+
+
+class AmbiguousColumn(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42702 - ambiguous column."""
+
+ pass
+
+
+class AmbiguousFunction(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42725 - ambiguous function."""
+
+ pass
+
+
+class AmbiguousParameter(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P08 - ambiguous parameter."""
+
+ pass
+
+
+class AmbiguousAlias(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P09 - ambiguous alias."""
+
+ pass
+
+
+class InvalidColumnReference(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P10 - invalid column reference."""
+
+ pass
+
+
+class InvalidColumnDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42611 - invalid column definition."""
+
+ pass
+
+
+class InvalidCursorDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P11 - invalid cursor definition."""
+
+ pass
+
+
+class InvalidDatabaseDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P12 - invalid database definition."""
+
+ pass
+
+
+class InvalidFunctionDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P13 - invalid function definition."""
+
+ pass
+
+
+class InvalidPreparedStatementDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P14 - invalid prepared statement definition."""
+
+ pass
+
+
+class InvalidSchemaDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P15 - invalid schema definition."""
+
+ pass
+
+
+class InvalidTableDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P16 - invalid table definition."""
+
+ pass
+
+
+class InvalidObjectDefinition(SyntaxErrorOrAccessRuleViolation):
+ """SQLSTATE 42P17 - invalid object definition."""
+
+ pass
+
+
+class WithCheckOptionViolation(LibpqError):
+ """SQLSTATE 44000 - with check option violation."""
+
+ pass
+
+
+class InsufficientResources(LibpqError):
+ """SQLSTATE 53000 - insufficient resources."""
+
+ pass
+
+
+class DiskFull(InsufficientResources):
+ """SQLSTATE 53100 - disk full."""
+
+ pass
+
+
+class OutOfMemory(InsufficientResources):
+ """SQLSTATE 53200 - out of memory."""
+
+ pass
+
+
+class TooManyConnections(InsufficientResources):
+ """SQLSTATE 53300 - too many connections."""
+
+ pass
+
+
+class ConfigurationLimitExceeded(InsufficientResources):
+ """SQLSTATE 53400 - configuration limit exceeded."""
+
+ pass
+
+
+class ProgramLimitExceeded(LibpqError):
+ """SQLSTATE 54000 - program limit exceeded."""
+
+ pass
+
+
+class StatementTooComplex(ProgramLimitExceeded):
+ """SQLSTATE 54001 - statement too complex."""
+
+ pass
+
+
+class TooManyColumns(ProgramLimitExceeded):
+ """SQLSTATE 54011 - too many columns."""
+
+ pass
+
+
+class TooManyArguments(ProgramLimitExceeded):
+ """SQLSTATE 54023 - too many arguments."""
+
+ pass
+
+
+class ObjectNotInPrerequisiteState(LibpqError):
+ """SQLSTATE 55000 - object not in prerequisite state."""
+
+ pass
+
+
+class ObjectInUse(ObjectNotInPrerequisiteState):
+ """SQLSTATE 55006 - object in use."""
+
+ pass
+
+
+class CantChangeRuntimeParam(ObjectNotInPrerequisiteState):
+ """SQLSTATE 55P02 - cant change runtime param."""
+
+ pass
+
+
+class LockNotAvailable(ObjectNotInPrerequisiteState):
+ """SQLSTATE 55P03 - lock not available."""
+
+ pass
+
+
+class UnsafeNewEnumValueUsage(ObjectNotInPrerequisiteState):
+ """SQLSTATE 55P04 - unsafe new enum value usage."""
+
+ pass
+
+
+class OperatorIntervention(LibpqError):
+ """SQLSTATE 57000 - operator intervention."""
+
+ pass
+
+
+class QueryCanceled(OperatorIntervention):
+ """SQLSTATE 57014 - query canceled."""
+
+ pass
+
+
+class AdminShutdown(OperatorIntervention):
+ """SQLSTATE 57P01 - admin shutdown."""
+
+ pass
+
+
+class CrashShutdown(OperatorIntervention):
+ """SQLSTATE 57P02 - crash shutdown."""
+
+ pass
+
+
+class CannotConnectNow(OperatorIntervention):
+ """SQLSTATE 57P03 - cannot connect now."""
+
+ pass
+
+
+class DatabaseDropped(OperatorIntervention):
+ """SQLSTATE 57P04 - database dropped."""
+
+ pass
+
+
+class IdleSessionTimeout(OperatorIntervention):
+ """SQLSTATE 57P05 - idle session timeout."""
+
+ pass
+
+
+class SystemError(LibpqError):
+ """SQLSTATE 58000 - system error."""
+
+ pass
+
+
+class IoError(SystemError):
+ """SQLSTATE 58030 - io error."""
+
+ pass
+
+
+class UndefinedFile(SystemError):
+ """SQLSTATE 58P01 - undefined file."""
+
+ pass
+
+
+class DuplicateFile(SystemError):
+ """SQLSTATE 58P02 - duplicate file."""
+
+ pass
+
+
+class FileNameTooLong(SystemError):
+ """SQLSTATE 58P03 - file name too long."""
+
+ pass
+
+
+class ConfigFileError(LibpqError):
+ """SQLSTATE F0000 - config file error."""
+
+ pass
+
+
+class LockFileExists(ConfigFileError):
+ """SQLSTATE F0001 - lock file exists."""
+
+ pass
+
+
+class FDWError(LibpqError):
+ """SQLSTATE HV000 - fdw error."""
+
+ pass
+
+
+class FDWColumnNameNotFound(FDWError):
+ """SQLSTATE HV005 - fdw column name not found."""
+
+ pass
+
+
+class FDWDynamicParameterValueNeeded(FDWError):
+ """SQLSTATE HV002 - fdw dynamic parameter value needed."""
+
+ pass
+
+
+class FDWFunctionSequenceError(FDWError):
+ """SQLSTATE HV010 - fdw function sequence error."""
+
+ pass
+
+
+class FDWInconsistentDescriptorInformation(FDWError):
+ """SQLSTATE HV021 - fdw inconsistent descriptor information."""
+
+ pass
+
+
+class FDWInvalidAttributeValue(FDWError):
+ """SQLSTATE HV024 - fdw invalid attribute value."""
+
+ pass
+
+
+class FDWInvalidColumnName(FDWError):
+ """SQLSTATE HV007 - fdw invalid column name."""
+
+ pass
+
+
+class FDWInvalidColumnNumber(FDWError):
+ """SQLSTATE HV008 - fdw invalid column number."""
+
+ pass
+
+
+class FDWInvalidDataType(FDWError):
+ """SQLSTATE HV004 - fdw invalid data type."""
+
+ pass
+
+
+class FDWInvalidDataTypeDescriptors(FDWError):
+ """SQLSTATE HV006 - fdw invalid data type descriptors."""
+
+ pass
+
+
+class FDWInvalidDescriptorFieldIdentifier(FDWError):
+ """SQLSTATE HV091 - fdw invalid descriptor field identifier."""
+
+ pass
+
+
+class FDWInvalidHandle(FDWError):
+ """SQLSTATE HV00B - fdw invalid handle."""
+
+ pass
+
+
+class FDWInvalidOptionIndex(FDWError):
+ """SQLSTATE HV00C - fdw invalid option index."""
+
+ pass
+
+
+class FDWInvalidOptionName(FDWError):
+ """SQLSTATE HV00D - fdw invalid option name."""
+
+ pass
+
+
+class FDWInvalidStringLengthOrBufferLength(FDWError):
+ """SQLSTATE HV090 - fdw invalid string length or buffer length."""
+
+ pass
+
+
+class FDWInvalidStringFormat(FDWError):
+ """SQLSTATE HV00A - fdw invalid string format."""
+
+ pass
+
+
+class FDWInvalidUseOfNullPointer(FDWError):
+ """SQLSTATE HV009 - fdw invalid use of null pointer."""
+
+ pass
+
+
+class FDWTooManyHandles(FDWError):
+ """SQLSTATE HV014 - fdw too many handles."""
+
+ pass
+
+
+class FDWOutOfMemory(FDWError):
+ """SQLSTATE HV001 - fdw out of memory."""
+
+ pass
+
+
+class FDWNoSchemas(FDWError):
+ """SQLSTATE HV00P - fdw no schemas."""
+
+ pass
+
+
+class FDWOptionNameNotFound(FDWError):
+ """SQLSTATE HV00J - fdw option name not found."""
+
+ pass
+
+
+class FDWReplyHandle(FDWError):
+ """SQLSTATE HV00K - fdw reply handle."""
+
+ pass
+
+
+class FDWSchemaNotFound(FDWError):
+ """SQLSTATE HV00Q - fdw schema not found."""
+
+ pass
+
+
+class FDWTableNotFound(FDWError):
+ """SQLSTATE HV00R - fdw table not found."""
+
+ pass
+
+
+class FDWUnableToCreateExecution(FDWError):
+ """SQLSTATE HV00L - fdw unable to create execution."""
+
+ pass
+
+
+class FDWUnableToCreateReply(FDWError):
+ """SQLSTATE HV00M - fdw unable to create reply."""
+
+ pass
+
+
+class FDWUnableToEstablishConnection(FDWError):
+ """SQLSTATE HV00N - fdw unable to establish connection."""
+
+ pass
+
+
+class PlpgsqlError(LibpqError):
+ """SQLSTATE P0000 - plpgsql error."""
+
+ pass
+
+
+class RaiseException(PlpgsqlError):
+ """SQLSTATE P0001 - raise exception."""
+
+ pass
+
+
+class NoDataFound(PlpgsqlError):
+ """SQLSTATE P0002 - no data found."""
+
+ pass
+
+
+class TooManyRows(PlpgsqlError):
+ """SQLSTATE P0003 - too many rows."""
+
+ pass
+
+
+class AssertFailure(PlpgsqlError):
+ """SQLSTATE P0004 - assert failure."""
+
+ pass
+
+
+class InternalError(LibpqError):
+ """SQLSTATE XX000 - internal error."""
+
+ pass
+
+
+class DataCorrupted(InternalError):
+ """SQLSTATE XX001 - data corrupted."""
+
+ pass
+
+
+class IndexCorrupted(InternalError):
+ """SQLSTATE XX002 - index corrupted."""
+
+ pass
+
+
+SQLSTATE_TO_EXCEPTION: Dict[str, type] = {
+ "00000": SuccessfulCompletion,
+ "01000": Warning,
+ "0100C": DynamicResultSetsReturnedWarning,
+ "01008": ImplicitZeroBitPaddingWarning,
+ "01003": NullValueEliminatedInSetFunctionWarning,
+ "01007": PrivilegeNotGrantedWarning,
+ "01006": PrivilegeNotRevokedWarning,
+ "01004": StringDataRightTruncationWarning,
+ "01P01": DeprecatedFeatureWarning,
+ "02000": NoData,
+ "02001": NoAdditionalDynamicResultSetsReturned,
+ "03000": SQLStatementNotYetComplete,
+ "08000": ConnectionException,
+ "08003": ConnectionDoesNotExist,
+ "08006": ConnectionFailure,
+ "08001": SQLClientUnableToEstablishSQLConnection,
+ "08004": SQLServerRejectedEstablishmentOfSQLConnection,
+ "08007": TransactionResolutionUnknown,
+ "08P01": ProtocolViolation,
+ "09000": TriggeredActionException,
+ "0A000": FeatureNotSupported,
+ "0B000": InvalidTransactionInitiation,
+ "0F000": LocatorException,
+ "0F001": InvalidLocatorSpecification,
+ "0L000": InvalidGrantor,
+ "0LP01": InvalidGrantOperation,
+ "0P000": InvalidRoleSpecification,
+ "0Z000": DiagnosticsException,
+ "0Z002": StackedDiagnosticsAccessedWithoutActiveHandler,
+ "10608": InvalidArgumentForXquery,
+ "20000": CaseNotFound,
+ "21000": CardinalityViolation,
+ "22000": DataException,
+ "2202E": ArraySubscriptError,
+ "22021": CharacterNotInRepertoire,
+ "22008": DatetimeFieldOverflow,
+ "22012": DivisionByZero,
+ "22005": ErrorInAssignment,
+ "2200B": EscapeCharacterConflict,
+ "22022": IndicatorOverflow,
+ "22015": IntervalFieldOverflow,
+ "2201E": InvalidArgumentForLogarithm,
+ "22014": InvalidArgumentForNtileFunction,
+ "22016": InvalidArgumentForNthValueFunction,
+ "2201F": InvalidArgumentForPowerFunction,
+ "2201G": InvalidArgumentForWidthBucketFunction,
+ "22018": InvalidCharacterValueForCast,
+ "22007": InvalidDatetimeFormat,
+ "22019": InvalidEscapeCharacter,
+ "2200D": InvalidEscapeOctet,
+ "22025": InvalidEscapeSequence,
+ "22P06": NonstandardUseOfEscapeCharacter,
+ "22010": InvalidIndicatorParameterValue,
+ "22023": InvalidParameterValue,
+ "22013": InvalidPrecedingOrFollowingSize,
+ "2201B": InvalidRegularExpression,
+ "2201W": InvalidRowCountInLimitClause,
+ "2201X": InvalidRowCountInResultOffsetClause,
+ "2202H": InvalidTablesampleArgument,
+ "2202G": InvalidTablesampleRepeat,
+ "22009": InvalidTimeZoneDisplacementValue,
+ "2200C": InvalidUseOfEscapeCharacter,
+ "2200G": MostSpecificTypeMismatch,
+ "22004": NullValueNotAllowed,
+ "22002": NullValueNoIndicatorParameter,
+ "22003": NumericValueOutOfRange,
+ "2200H": SequenceGeneratorLimitExceeded,
+ "22026": StringDataLengthMismatch,
+ "22001": StringDataRightTruncation,
+ "22011": SubstringError,
+ "22027": TrimError,
+ "22024": UnterminatedCString,
+ "2200F": ZeroLengthCharacterString,
+ "22P01": FloatingPointException,
+ "22P02": InvalidTextRepresentation,
+ "22P03": InvalidBinaryRepresentation,
+ "22P04": BadCopyFileFormat,
+ "22P05": UntranslatableCharacter,
+ "2200L": NotAnXmlDocument,
+ "2200M": InvalidXmlDocument,
+ "2200N": InvalidXmlContent,
+ "2200S": InvalidXmlComment,
+ "2200T": InvalidXmlProcessingInstruction,
+ "22030": DuplicateJsonObjectKeyValue,
+ "22031": InvalidArgumentForSQLJsonDatetimeFunction,
+ "22032": InvalidJsonText,
+ "22033": InvalidSQLJsonSubscript,
+ "22034": MoreThanOneSQLJsonItem,
+ "22035": NoSQLJsonItem,
+ "22036": NonNumericSQLJsonItem,
+ "22037": NonUniqueKeysInAJsonObject,
+ "22038": SingletonSQLJsonItemRequired,
+ "22039": SQLJsonArrayNotFound,
+ "2203A": SQLJsonMemberNotFound,
+ "2203B": SQLJsonNumberNotFound,
+ "2203C": SQLJsonObjectNotFound,
+ "2203D": TooManyJsonArrayElements,
+ "2203E": TooManyJsonObjectMembers,
+ "2203F": SQLJsonScalarRequired,
+ "2203G": SQLJsonItemCannotBeCastToTargetType,
+ "23000": IntegrityConstraintViolation,
+ "23001": RestrictViolation,
+ "23502": NotNullViolation,
+ "23503": ForeignKeyViolation,
+ "23505": UniqueViolation,
+ "23514": CheckViolation,
+ "23P01": ExclusionViolation,
+ "24000": InvalidCursorState,
+ "25000": InvalidTransactionState,
+ "25001": ActiveSQLTransaction,
+ "25002": BranchTransactionAlreadyActive,
+ "25008": HeldCursorRequiresSameIsolationLevel,
+ "25003": InappropriateAccessModeForBranchTransaction,
+ "25004": InappropriateIsolationLevelForBranchTransaction,
+ "25005": NoActiveSQLTransactionForBranchTransaction,
+ "25006": ReadOnlySQLTransaction,
+ "25007": SchemaAndDataStatementMixingNotSupported,
+ "25P01": NoActiveSQLTransaction,
+ "25P02": InFailedSQLTransaction,
+ "25P03": IdleInTransactionSessionTimeout,
+ "25P04": TransactionTimeout,
+ "26000": InvalidSQLStatementName,
+ "27000": TriggeredDataChangeViolation,
+ "28000": InvalidAuthorizationSpecification,
+ "28P01": InvalidPassword,
+ "2B000": DependentPrivilegeDescriptorsStillExist,
+ "2BP01": DependentObjectsStillExist,
+ "2D000": InvalidTransactionTermination,
+ "2F000": SQLRoutineException,
+ "2F005": FunctionExecutedNoReturnStatement,
+ "2F002": SREModifyingSQLDataNotPermitted,
+ "2F003": SREProhibitedSQLStatementAttempted,
+ "2F004": SREReadingSQLDataNotPermitted,
+ "34000": InvalidCursorName,
+ "38000": ExternalRoutineException,
+ "38001": ContainingSQLNotPermitted,
+ "38002": EREModifyingSQLDataNotPermitted,
+ "38003": EREProhibitedSQLStatementAttempted,
+ "38004": EREReadingSQLDataNotPermitted,
+ "39000": ExternalRoutineInvocationException,
+ "39001": InvalidSqlstateReturned,
+ "39004": ERIENullValueNotAllowed,
+ "39P01": TriggerProtocolViolated,
+ "39P02": SrfProtocolViolated,
+ "39P03": EventTriggerProtocolViolated,
+ "3B000": SavepointException,
+ "3B001": InvalidSavepointSpecification,
+ "3D000": InvalidCatalogName,
+ "3F000": InvalidSchemaName,
+ "40000": TransactionRollback,
+ "40002": TransactionIntegrityConstraintViolation,
+ "40001": SerializationFailure,
+ "40003": StatementCompletionUnknown,
+ "40P01": DeadlockDetected,
+ "42000": SyntaxErrorOrAccessRuleViolation,
+ "42601": SyntaxError,
+ "42501": InsufficientPrivilege,
+ "42846": CannotCoerce,
+ "42803": GroupingError,
+ "42P20": WindowingError,
+ "42P19": InvalidRecursion,
+ "42830": InvalidForeignKey,
+ "42602": InvalidName,
+ "42622": NameTooLong,
+ "42939": ReservedName,
+ "42804": DatatypeMismatch,
+ "42P18": IndeterminateDatatype,
+ "42P21": CollationMismatch,
+ "42P22": IndeterminateCollation,
+ "42809": WrongObjectType,
+ "428C9": GeneratedAlways,
+ "42703": UndefinedColumn,
+ "42883": UndefinedFunction,
+ "42P01": UndefinedTable,
+ "42P02": UndefinedParameter,
+ "42704": UndefinedObject,
+ "42701": DuplicateColumn,
+ "42P03": DuplicateCursor,
+ "42P04": DuplicateDatabase,
+ "42723": DuplicateFunction,
+ "42P05": DuplicatePreparedStatement,
+ "42P06": DuplicateSchema,
+ "42P07": DuplicateTable,
+ "42712": DuplicateAlias,
+ "42710": DuplicateObject,
+ "42702": AmbiguousColumn,
+ "42725": AmbiguousFunction,
+ "42P08": AmbiguousParameter,
+ "42P09": AmbiguousAlias,
+ "42P10": InvalidColumnReference,
+ "42611": InvalidColumnDefinition,
+ "42P11": InvalidCursorDefinition,
+ "42P12": InvalidDatabaseDefinition,
+ "42P13": InvalidFunctionDefinition,
+ "42P14": InvalidPreparedStatementDefinition,
+ "42P15": InvalidSchemaDefinition,
+ "42P16": InvalidTableDefinition,
+ "42P17": InvalidObjectDefinition,
+ "44000": WithCheckOptionViolation,
+ "53000": InsufficientResources,
+ "53100": DiskFull,
+ "53200": OutOfMemory,
+ "53300": TooManyConnections,
+ "53400": ConfigurationLimitExceeded,
+ "54000": ProgramLimitExceeded,
+ "54001": StatementTooComplex,
+ "54011": TooManyColumns,
+ "54023": TooManyArguments,
+ "55000": ObjectNotInPrerequisiteState,
+ "55006": ObjectInUse,
+ "55P02": CantChangeRuntimeParam,
+ "55P03": LockNotAvailable,
+ "55P04": UnsafeNewEnumValueUsage,
+ "57000": OperatorIntervention,
+ "57014": QueryCanceled,
+ "57P01": AdminShutdown,
+ "57P02": CrashShutdown,
+ "57P03": CannotConnectNow,
+ "57P04": DatabaseDropped,
+ "57P05": IdleSessionTimeout,
+ "58000": SystemError,
+ "58030": IoError,
+ "58P01": UndefinedFile,
+ "58P02": DuplicateFile,
+ "58P03": FileNameTooLong,
+ "F0000": ConfigFileError,
+ "F0001": LockFileExists,
+ "HV000": FDWError,
+ "HV005": FDWColumnNameNotFound,
+ "HV002": FDWDynamicParameterValueNeeded,
+ "HV010": FDWFunctionSequenceError,
+ "HV021": FDWInconsistentDescriptorInformation,
+ "HV024": FDWInvalidAttributeValue,
+ "HV007": FDWInvalidColumnName,
+ "HV008": FDWInvalidColumnNumber,
+ "HV004": FDWInvalidDataType,
+ "HV006": FDWInvalidDataTypeDescriptors,
+ "HV091": FDWInvalidDescriptorFieldIdentifier,
+ "HV00B": FDWInvalidHandle,
+ "HV00C": FDWInvalidOptionIndex,
+ "HV00D": FDWInvalidOptionName,
+ "HV090": FDWInvalidStringLengthOrBufferLength,
+ "HV00A": FDWInvalidStringFormat,
+ "HV009": FDWInvalidUseOfNullPointer,
+ "HV014": FDWTooManyHandles,
+ "HV001": FDWOutOfMemory,
+ "HV00P": FDWNoSchemas,
+ "HV00J": FDWOptionNameNotFound,
+ "HV00K": FDWReplyHandle,
+ "HV00Q": FDWSchemaNotFound,
+ "HV00R": FDWTableNotFound,
+ "HV00L": FDWUnableToCreateExecution,
+ "HV00M": FDWUnableToCreateReply,
+ "HV00N": FDWUnableToEstablishConnection,
+ "P0000": PlpgsqlError,
+ "P0001": RaiseException,
+ "P0002": NoDataFound,
+ "P0003": TooManyRows,
+ "P0004": AssertFailure,
+ "XX000": InternalError,
+ "XX001": DataCorrupted,
+ "XX002": IndexCorrupted,
+}
+
+
+__all__ = [
+ "InvalidCursorName",
+ "UndefinedParameter",
+ "UndefinedColumn",
+ "NotAnXmlDocument",
+ "FDWOutOfMemory",
+ "InvalidRoleSpecification",
+ "InvalidArgumentForNthValueFunction",
+ "SQLJsonObjectNotFound",
+ "FDWSchemaNotFound",
+ "InvalidParameterValue",
+ "InvalidTableDefinition",
+ "AssertFailure",
+ "FDWInvalidOptionName",
+ "InvalidEscapeOctet",
+ "ReadOnlySQLTransaction",
+ "ExternalRoutineInvocationException",
+ "CrashShutdown",
+ "FDWInvalidOptionIndex",
+ "NotNullViolation",
+ "ConfigFileError",
+ "InvalidSQLJsonSubscript",
+ "InvalidForeignKey",
+ "InsufficientResources",
+ "ObjectNotInPrerequisiteState",
+ "InvalidRowCountInLimitClause",
+ "IntervalFieldOverflow",
+ "CollationMismatch",
+ "InvalidArgumentForNtileFunction",
+ "InvalidCharacterValueForCast",
+ "NonUniqueKeysInAJsonObject",
+ "DependentPrivilegeDescriptorsStillExist",
+ "InFailedSQLTransaction",
+ "GroupingError",
+ "TransactionTimeout",
+ "CaseNotFound",
+ "ConnectionException",
+ "DuplicateJsonObjectKeyValue",
+ "InvalidSchemaDefinition",
+ "FDWUnableToCreateReply",
+ "UndefinedTable",
+ "SequenceGeneratorLimitExceeded",
+ "InvalidJsonText",
+ "IdleSessionTimeout",
+ "NullValueNotAllowed",
+ "BranchTransactionAlreadyActive",
+ "InvalidGrantOperation",
+ "NullValueNoIndicatorParameter",
+ "ProtocolViolation",
+ "FDWInvalidDataTypeDescriptors",
+ "TriggeredDataChangeViolation",
+ "ExternalRoutineException",
+ "InvalidSqlstateReturned",
+ "PlpgsqlError",
+ "InvalidXmlContent",
+ "TriggeredActionException",
+ "SQLClientUnableToEstablishSQLConnection",
+ "FDWTableNotFound",
+ "NumericValueOutOfRange",
+ "RestrictViolation",
+ "AmbiguousParameter",
+ "StatementTooComplex",
+ "UnsafeNewEnumValueUsage",
+ "NonNumericSQLJsonItem",
+ "InvalidIndicatorParameterValue",
+ "ExclusionViolation",
+ "OperatorIntervention",
+ "QueryCanceled",
+ "Warning",
+ "InvalidArgumentForSQLJsonDatetimeFunction",
+ "ForeignKeyViolation",
+ "StringDataLengthMismatch",
+ "SQLRoutineException",
+ "TooManyConnections",
+ "TooManyJsonObjectMembers",
+ "NoData",
+ "UntranslatableCharacter",
+ "FDWUnableToEstablishConnection",
+ "LockFileExists",
+ "SREReadingSQLDataNotPermitted",
+ "IndeterminateDatatype",
+ "CheckViolation",
+ "InvalidDatabaseDefinition",
+ "NoActiveSQLTransactionForBranchTransaction",
+ "SQLServerRejectedEstablishmentOfSQLConnection",
+ "DuplicateFile",
+ "FDWInvalidColumnNumber",
+ "TransactionRollback",
+ "MoreThanOneSQLJsonItem",
+ "WithCheckOptionViolation",
+ "FDWNoSchemas",
+ "GeneratedAlways",
+ "CannotConnectNow",
+ "CardinalityViolation",
+ "InvalidAuthorizationSpecification",
+ "SQLJsonNumberNotFound",
+ "SQLJsonMemberNotFound",
+ "InvalidUseOfEscapeCharacter",
+ "UnterminatedCString",
+ "TrimError",
+ "SrfProtocolViolated",
+ "DiskFull",
+ "TooManyColumns",
+ "InvalidObjectDefinition",
+ "InvalidArgumentForLogarithm",
+ "TooManyJsonArrayElements",
+ "OutOfMemory",
+ "EREProhibitedSQLStatementAttempted",
+ "FDWInvalidStringFormat",
+ "StackedDiagnosticsAccessedWithoutActiveHandler",
+ "SchemaAndDataStatementMixingNotSupported",
+ "InternalError",
+ "InvalidEscapeCharacter",
+ "FDWError",
+ "ImplicitZeroBitPaddingWarning",
+ "DivisionByZero",
+ "InvalidTablesampleArgument",
+ "DeadlockDetected",
+ "CantChangeRuntimeParam",
+ "UndefinedObject",
+ "UniqueViolation",
+ "InvalidCursorDefinition",
+ "ConnectionFailure",
+ "UndefinedFunction",
+ "FDWFunctionSequenceError",
+ "ErrorInAssignment",
+ "SuccessfulCompletion",
+ "StringDataRightTruncation",
+ "FDWTooManyHandles",
+ "FDWInvalidDataType",
+ "ActiveSQLTransaction",
+ "InvalidTextRepresentation",
+ "InvalidSQLStatementName",
+ "PrivilegeNotGrantedWarning",
+ "SREModifyingSQLDataNotPermitted",
+ "IndeterminateCollation",
+ "SystemError",
+ "NullValueEliminatedInSetFunctionWarning",
+ "DependentObjectsStillExist",
+ "InvalidSchemaName",
+ "DuplicateColumn",
+ "FunctionExecutedNoReturnStatement",
+ "InvalidColumnDefinition",
+ "DynamicResultSetsReturnedWarning",
+ "IdleInTransactionSessionTimeout",
+ "StatementCompletionUnknown",
+ "CannotCoerce",
+ "InvalidTransactionState",
+ "DuplicateTable",
+ "BadCopyFileFormat",
+ "ZeroLengthCharacterString",
+ "SyntaxErrorOrAccessRuleViolation",
+ "SingletonSQLJsonItemRequired",
+ "IndexCorrupted",
+ "FDWInvalidColumnName",
+ "DataCorrupted",
+ "ERIENullValueNotAllowed",
+ "ArraySubscriptError",
+ "FDWReplyHandle",
+ "DiagnosticsException",
+ "InvalidTablesampleRepeat",
+ "SQLJsonItemCannotBeCastToTargetType",
+ "FDWInvalidHandle",
+ "InvalidPassword",
+ "InvalidEscapeSequence",
+ "EscapeCharacterConflict",
+ "InvalidSavepointSpecification",
+ "FDWInvalidAttributeValue",
+ "ContainingSQLNotPermitted",
+ "LocatorException",
+ "DatatypeMismatch",
+ "InvalidCursorState",
+ "InvalidName",
+ "IndicatorOverflow",
+ "ReservedName",
+ "DatetimeFieldOverflow",
+ "FDWInconsistentDescriptorInformation",
+ "FloatingPointException",
+ "AmbiguousAlias",
+ "InvalidRecursion",
+ "WrongObjectType",
+ "UndefinedFile",
+ "LockNotAvailable",
+ "InvalidRowCountInResultOffsetClause",
+ "ObjectInUse",
+ "DeprecatedFeatureWarning",
+ "FDWDynamicParameterValueNeeded",
+ "DuplicateFunction",
+ "InvalidXmlDocument",
+ "StringDataRightTruncationWarning",
+ "DuplicatePreparedStatement",
+ "InvalidGrantor",
+ "EventTriggerProtocolViolated",
+ "FDWInvalidUseOfNullPointer",
+ "FDWUnableToCreateExecution",
+ "ConnectionDoesNotExist",
+ "InvalidCatalogName",
+ "InvalidArgumentForXquery",
+ "FDWColumnNameNotFound",
+ "TransactionIntegrityConstraintViolation",
+ "InvalidPreparedStatementDefinition",
+ "FDWInvalidDescriptorFieldIdentifier",
+ "FDWOptionNameNotFound",
+ "InvalidArgumentForPowerFunction",
+ "FDWInvalidStringLengthOrBufferLength",
+ "SREProhibitedSQLStatementAttempted",
+ "NoDataFound",
+ "DuplicateDatabase",
+ "FeatureNotSupported",
+ "IntegrityConstraintViolation",
+ "AmbiguousColumn",
+ "PrivilegeNotRevokedWarning",
+ "FileNameTooLong",
+ "InvalidArgumentForWidthBucketFunction",
+ "HeldCursorRequiresSameIsolationLevel",
+ "NoSQLJsonItem",
+ "IoError",
+ "SavepointException",
+ "NoActiveSQLTransaction",
+ "InvalidFunctionDefinition",
+ "AdminShutdown",
+ "DatabaseDropped",
+ "InvalidRegularExpression",
+ "WindowingError",
+ "InvalidColumnReference",
+ "InvalidBinaryRepresentation",
+ "SQLJsonScalarRequired",
+ "ConfigurationLimitExceeded",
+ "SyntaxError",
+ "SerializationFailure",
+ "ProgramLimitExceeded",
+ "DuplicateSchema",
+ "SQLStatementNotYetComplete",
+ "LibpqError",
+ "DataException",
+ "SubstringError",
+ "InvalidLocatorSpecification",
+ "InappropriateAccessModeForBranchTransaction",
+ "EREModifyingSQLDataNotPermitted",
+ "InsufficientPrivilege",
+ "NoAdditionalDynamicResultSetsReturned",
+ "SQLJsonArrayNotFound",
+ "NameTooLong",
+ "InvalidTimeZoneDisplacementValue",
+ "InappropriateIsolationLevelForBranchTransaction",
+ "RaiseException",
+ "EREReadingSQLDataNotPermitted",
+ "TriggerProtocolViolated",
+ "NonstandardUseOfEscapeCharacter",
+ "InvalidTransactionInitiation",
+ "DuplicateAlias",
+ "TransactionResolutionUnknown",
+ "TooManyRows",
+ "InvalidXmlComment",
+ "MostSpecificTypeMismatch",
+ "DuplicateObject",
+ "DuplicateCursor",
+ "AmbiguousFunction",
+ "TooManyArguments",
+ "InvalidXmlProcessingInstruction",
+ "InvalidTransactionTermination",
+ "InvalidDatetimeFormat",
+ "InvalidPrecedingOrFollowingSize",
+ "CharacterNotInRepertoire",
+ "SQLSTATE_TO_EXCEPTION",
+]
diff --git a/src/test/pytest/libpq/errors.py b/src/test/pytest/libpq/errors.py
new file mode 100644
index 00000000000..764a96c2478
--- /dev/null
+++ b/src/test/pytest/libpq/errors.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+PostgreSQL error types mapped from SQLSTATE codes.
+
+This module provides LibpqError and its subclasses for handling PostgreSQL
+errors based on SQLSTATE codes. The exception classes in _generated_errors.py
+are auto-generated from src/backend/utils/errcodes.txt.
+
+To regenerate: src/tools/generate_pytest_libpq_errors.py
+"""
+
+from typing import Optional
+
+from ._error_base import LibpqError, LibpqWarning
+from ._generated_errors import (
+ SQLSTATE_TO_EXCEPTION,
+)
+from ._generated_errors import * # noqa: F403
+
+
+def get_exception_class(sqlstate: Optional[str]) -> type:
+ """Get the appropriate exception class for a SQLSTATE code."""
+ if sqlstate in SQLSTATE_TO_EXCEPTION:
+ return SQLSTATE_TO_EXCEPTION[sqlstate]
+ return LibpqError
+
+
+def make_error(message: str, *, sqlstate: Optional[str] = None, **kwargs) -> LibpqError:
+ """Create an appropriate LibpqError subclass based on the SQLSTATE code."""
+ exc_class = get_exception_class(sqlstate)
+ return exc_class(message, sqlstate=sqlstate, **kwargs)
+
+
+__all__ = [
+ "LibpqError",
+ "LibpqWarning",
+ "make_error",
+]
diff --git a/src/test/pytest/meson.build b/src/test/pytest/meson.build
index abd128dfa24..b86be901e7c 100644
--- a/src/test/pytest/meson.build
+++ b/src/test/pytest/meson.build
@@ -10,7 +10,10 @@ tests += {
'bd': meson.current_build_dir(),
'pytest': {
'tests': [
- 'pyt/test_something.py',
+ 'pyt/test_errors.py',
+ 'pyt/test_libpq.py',
+ 'pyt/test_multi_server.py',
+ 'pyt/test_query_helpers.py',
],
},
}
diff --git a/src/test/pytest/pypg/__init__.py b/src/test/pytest/pypg/__init__.py
new file mode 100644
index 00000000000..4ee91289f70
--- /dev/null
+++ b/src/test/pytest/pypg/__init__.py
@@ -0,0 +1,10 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+from ._env import require_test_extras, skip_unless_test_extras
+from .server import PostgresServer
+
+__all__ = [
+ "require_test_extras",
+ "skip_unless_test_extras",
+ "PostgresServer",
+]
diff --git a/src/test/pytest/pypg/_env.py b/src/test/pytest/pypg/_env.py
new file mode 100644
index 00000000000..c4087be3212
--- /dev/null
+++ b/src/test/pytest/pypg/_env.py
@@ -0,0 +1,72 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import logging
+import os
+
+import pytest
+
+logger = logging.getLogger(__name__)
+
+
+def _test_extra_skip_reason(*keys: str) -> str:
+ return "requires {} to be set in PG_TEST_EXTRA".format(", ".join(keys))
+
+
+def _has_test_extra(key: str) -> bool:
+ """
+ Returns True if the PG_TEST_EXTRA environment variable contains the given
+ key.
+ """
+ extra = os.getenv("PG_TEST_EXTRA", "")
+ return key in extra.split()
+
+
+def require_test_extras(*keys: str):
+ """
+ A convenience annotation which will skip tests if all of the required keys
+ are not present in PG_TEST_EXTRA.
+
+ To skip a particular test function or class:
+
+ @pypg.require_test_extras("ldap")
+ def test_some_ldap_feature():
+ ...
+
+ To skip an entire module:
+
+ pytestmark = pypg.require_test_extra("ssl", "kerberos")
+ """
+ return pytest.mark.skipif(
+ not all([_has_test_extra(k) for k in keys]),
+ reason=_test_extra_skip_reason(*keys),
+ )
+
+
+def skip_unless_test_extras(*keys: str):
+ """
+ Skip the current test/fixture if any of the required keys are not present
+ in PG_TEST_EXTRA. Use this inside fixtures where decorators can't be used.
+
+ @pytest.fixture
+ def my_fixture():
+ skip_unless_test_extras("ldap")
+ ...
+ """
+ if not all([_has_test_extra(k) for k in keys]):
+ pytest.skip(_test_extra_skip_reason(*keys))
+
+
+def test_timeout_default() -> int:
+ """
+ Returns the value of the PG_TEST_TIMEOUT_DEFAULT environment variable, in
+ seconds, or 180 if one was not provided.
+ """
+ default = os.getenv("PG_TEST_TIMEOUT_DEFAULT", "")
+ if not default:
+ return 180
+
+ try:
+ return int(default)
+ except ValueError as v:
+ logger.warning("PG_TEST_TIMEOUT_DEFAULT could not be parsed: " + str(v))
+ return 180
diff --git a/src/test/pytest/pypg/fixtures.py b/src/test/pytest/pypg/fixtures.py
new file mode 100644
index 00000000000..8c0cb60daa5
--- /dev/null
+++ b/src/test/pytest/pypg/fixtures.py
@@ -0,0 +1,335 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import os
+import contextlib
+import pathlib
+import time
+from typing import List
+
+import pytest
+
+from ._env import test_timeout_default
+from .util import capture
+from .server import PostgresServer
+
+from libpq import load_libpq_handle, connect as libpq_connect
+
+
+# Stash key for tracking servers for log reporting.
+_servers_key = pytest.StashKey[List[PostgresServer]]()
+
+
+def _record_server_for_log_reporting(request, server):
+ """Record a server for log reporting on test failure."""
+ if _servers_key not in request.node.stash:
+ request.node.stash[_servers_key] = []
+ request.node.stash[_servers_key].append(server)
+
+
[email protected]
+def remaining_timeout():
+ """
+ This fixture provides a function that returns how much of the
+ PG_TEST_TIMEOUT_DEFAULT remains for the current test, in fractional seconds.
+ This value is never less than zero.
+
+ This fixture is per-test, so the deadline is also reset on a per-test basis.
+ """
+ now = time.monotonic()
+ deadline = now + test_timeout_default()
+
+ return lambda: max(deadline - time.monotonic(), 0)
+
+
[email protected](scope="module")
+def remaining_timeout_module():
+ """
+ Same as remaining_timeout, but the deadline is set once per module.
+
+ This fixture is per-module, which means it's generally only really useful
+ for configuring timeouts of operations that happen in the setup phase of
+ another module fixtures. If you use it in a test it would mean that each
+ subsequent test in the module gets a reduced timeout.
+ """
+ now = time.monotonic()
+ deadline = now + test_timeout_default()
+
+ return lambda: max(deadline - time.monotonic(), 0)
+
+
[email protected](scope="session")
+def libpq_handle(libdir, bindir):
+ """
+ Loads a ctypes handle for libpq. Some common function prototypes are
+ initialized for general use.
+ """
+ try:
+ return load_libpq_handle(libdir, bindir)
+ except OSError as e:
+ if "wrong ELF class" in str(e):
+ # This happens in CI when trying to lead a 32-bit libpq library
+ # with a 64-bit Python
+ pytest.skip("libpq architecture does not match Python interpreter")
+ raise
+
+
[email protected]
+def connect(libpq_handle, remaining_timeout):
+ """
+ Returns a function to connect to PostgreSQL via libpq.
+
+ The returned function accepts connection options as keyword arguments
+ (host, port, dbname, etc.) and returns a PGconn object. Connections
+ are automatically cleaned up at the end of the test.
+
+ Example:
+ conn = connect(host='localhost', port=5432, dbname='postgres')
+ result = conn.sql("SELECT 1")
+ """
+ with contextlib.ExitStack() as stack:
+
+ def _connect(**opts):
+ return libpq_connect(libpq_handle, stack, remaining_timeout, **opts)
+
+ yield _connect
+
+
[email protected](scope="session")
+def pg_config():
+ """
+ Returns the path to pg_config. Uses PG_CONFIG environment variable if set,
+ otherwise uses 'pg_config' from PATH.
+ """
+ return os.environ.get("PG_CONFIG", "pg_config")
+
+
[email protected](scope="session")
+def bindir(pg_config):
+ """
+ Returns the PostgreSQL bin directory using pg_config --bindir.
+ """
+ return pathlib.Path(capture(pg_config, "--bindir"))
+
+
[email protected](scope="session")
+def libdir(pg_config):
+ """
+ Returns the PostgreSQL lib directory using pg_config --libdir.
+ """
+ return pathlib.Path(capture(pg_config, "--libdir"))
+
+
[email protected](scope="session")
+def tmp_check(tmp_path_factory) -> pathlib.Path:
+ """
+ Returns the tmp_check directory that should be used for the tests. If
+ TESTDATADIR is provided, that will be used; otherwise a new temporary
+ directory is created in the pytest temp root.
+ """
+ d = os.getenv("TESTDATADIR")
+ if d:
+ d = pathlib.Path(d)
+ else:
+ d = tmp_path_factory.mktemp("tmp_check")
+
+ return d
+
+
[email protected](scope="session")
+def datadir(tmp_check):
+ """
+ Returns the data directory to use for the pg fixture.
+ """
+
+ return tmp_check / "pgdata"
+
+
[email protected](scope="session")
+def sockdir(tmp_path_factory):
+ """
+ Returns the directory name to use as the server's unix_socket_directories
+ setting. Local client connections use this as the PGHOST.
+
+ At the moment, this is always put under the pytest temp root.
+ """
+ return tmp_path_factory.mktemp("sockfiles")
+
+
[email protected](scope="session")
+def pg_server_global(bindir, datadir, sockdir, libpq_handle):
+ """
+ Starts a running Postgres server listening on localhost. The HBA initially
+ allows only local UNIX connections from the same user.
+
+ Returns a PostgresServer instance with methods for server management, configuration,
+ and creating test databases/users.
+ """
+ server = PostgresServer("default", bindir, datadir, sockdir, libpq_handle)
+
+ yield server
+
+ # Cleanup any test resources
+ server.cleanup()
+
+ # Stop the server
+ server.stop()
+
+
[email protected](scope="module")
+def pg_server_module(pg_server_global):
+ """
+ Module-scoped server context. Which can be useful so that certain settings
+ can be overriden at the module level through autouse fixtures. An example
+ of this is in the SSL tests.
+ """
+ with pg_server_global.subcontext() as s:
+ yield s
+
+
[email protected]
+def pg(request, pg_server_module, remaining_timeout):
+ """
+ Per-test server context. Use this fixture to make changes to the server
+ which will be rolled back at the end of the test (e.g., creating test
+ users/databases).
+
+ Also captures the PostgreSQL log position at test start so that any new
+ log entries can be included in the test report on failure.
+ """
+ with pg_server_module.start_new_test(remaining_timeout) as s:
+ _record_server_for_log_reporting(request, s)
+ yield s
+
+
[email protected]
+def conn(pg):
+ """
+ Returns a connected PGconn instance to the test PostgreSQL server.
+ The connection is automatically cleaned up at the end of the test.
+
+ Example:
+ def test_something(conn):
+ result = conn.sql("SELECT 1")
+ assert result == 1
+ """
+ return pg.connect()
+
+
[email protected]
+def create_pg(request, bindir, sockdir, libpq_handle, tmp_check, remaining_timeout):
+ """
+ Factory fixture to create additional PostgreSQL servers (per-test scope).
+
+ Returns a function that creates new PostgreSQL server instances.
+ Servers are automatically cleaned up at the end of the test.
+
+ Example:
+ def test_multiple_servers(create_pg):
+ node1 = create_pg()
+ node2 = create_pg()
+ node3 = create_pg()
+ """
+ servers = []
+
+ def _create(name=None, **kwargs):
+ if name is None:
+ count = len(servers) + 1
+ name = f"pg{count}"
+
+ datadir = tmp_check / f"pgdata_{name}"
+ server = PostgresServer(name, bindir, datadir, sockdir, libpq_handle, **kwargs)
+ server.set_timeout(remaining_timeout)
+ _record_server_for_log_reporting(request, server)
+ servers.append(server)
+ return server
+
+ yield _create
+
+ for server in servers:
+ server.cleanup()
+ server.stop()
+
+
[email protected](scope="module")
+def _module_scoped_servers():
+ """Session-scoped list to track servers created by create_pg_module."""
+ return []
+
+
[email protected](scope="module")
+def create_pg_module(
+ bindir,
+ sockdir,
+ libpq_handle,
+ tmp_check,
+ remaining_timeout_module,
+ _module_scoped_servers,
+):
+ """
+ Factory fixture to create additional PostgreSQL servers (module scope).
+
+ Like create_pg, but servers persist for the entire test module.
+ Use this when multiple tests in a module can share the same servers.
+
+ The timeout is automatically set on all servers at the start of each test
+ via the _set_module_server_timeouts autouse fixture.
+
+ Example:
+ @pytest.fixture(scope="module")
+ def shared_nodes(create_pg_module):
+ return [create_pg_module() for _ in range(3)]
+ """
+
+ def _create(name=None, **kwargs):
+ if name is None:
+ count = len(_module_scoped_servers) + 1
+ name = f"pg{count}"
+ datadir = tmp_check / f"pgdata_{name}"
+ server = PostgresServer(name, bindir, datadir, sockdir, libpq_handle, **kwargs)
+ server.set_timeout(remaining_timeout_module)
+ _module_scoped_servers.append(server)
+ return server
+
+ yield _create
+
+ for server in _module_scoped_servers:
+ server.cleanup()
+ server.stop()
+
+
[email protected](autouse=True)
+def _set_module_server_timeouts(request, _module_scoped_servers, remaining_timeout):
+ """Autouse fixture that sets timeout, enters subcontext, and records log positions for module-scoped servers."""
+ with contextlib.ExitStack() as stack:
+ for server in _module_scoped_servers:
+ stack.enter_context(server.start_new_test(remaining_timeout))
+ _record_server_for_log_reporting(request, server)
+ yield
+
+
[email protected](hookwrapper=True, trylast=True)
+def pytest_runtest_makereport(item, call):
+ """
+ Adds PostgreSQL server logs to the test report sections.
+ """
+ outcome = yield
+ report = outcome.get_result()
+
+ if report.when != "call":
+ return
+
+ if _servers_key not in item.stash:
+ return
+
+ servers = item.stash[_servers_key]
+ del item.stash[_servers_key]
+
+ include_name = len(servers) > 1
+
+ for server in servers:
+ content = server.log_content()
+ if content.strip():
+ section_title = "Postgres log"
+ if include_name:
+ section_title += f" ({server.name})"
+ report.sections.append((section_title, content))
diff --git a/src/test/pytest/pypg/server.py b/src/test/pytest/pypg/server.py
new file mode 100644
index 00000000000..9242ab25007
--- /dev/null
+++ b/src/test/pytest/pypg/server.py
@@ -0,0 +1,470 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import contextlib
+import os
+import pathlib
+import platform
+import re
+import shutil
+import socket
+import subprocess
+import tempfile
+from collections import namedtuple
+from typing import Callable, Optional
+
+from .util import run
+from libpq import PGconn, connect as libpq_connect
+
+
+class FileBackup(contextlib.AbstractContextManager):
+ """
+ A context manager which backs up a file's contents, restoring them on exit.
+ """
+
+ def __init__(self, file: pathlib.Path):
+ super().__init__()
+
+ self._file = file
+
+ def __enter__(self):
+ with tempfile.NamedTemporaryFile(
+ prefix=self._file.name, dir=self._file.parent, delete=False
+ ) as f:
+ self._backup = pathlib.Path(f.name)
+
+ shutil.copyfile(self._file, self._backup)
+
+ return self
+
+ def __exit__(self, *exc):
+ # Swap the backup and the original file, so that the modified contents
+ # can still be inspected in case of failure.
+ tmp = self._backup.parent / (self._backup.name + ".tmp")
+
+ shutil.copyfile(self._file, tmp)
+ shutil.copyfile(self._backup, self._file)
+ shutil.move(tmp, self._backup)
+
+
+class HBA(FileBackup):
+ """
+ Backs up a server's HBA configuration and provides means for temporarily
+ editing it.
+ """
+
+ def __init__(self, datadir: pathlib.Path):
+ super().__init__(datadir / "pg_hba.conf")
+
+ def prepend(self, *lines):
+ """
+ Temporarily prepends lines to the server's pg_hba.conf.
+
+ As sugar for aligning HBA columns in the tests, each line can be either
+ a string or a list of strings. List elements will be joined by single
+ spaces before they are written to file.
+ """
+ with open(self._file, "r") as f:
+ prior_data = f.read()
+
+ with open(self._file, "w") as f:
+ for line in lines:
+ if isinstance(line, list):
+ print(*line, file=f)
+ else:
+ print(line, file=f)
+
+ f.write(prior_data)
+
+
+class Config(FileBackup):
+ """
+ Backs up a server's postgresql.conf and provides means for temporarily
+ editing it.
+ """
+
+ def __init__(self, datadir: pathlib.Path):
+ super().__init__(datadir / "postgresql.conf")
+
+ def set(self, **gucs):
+ """
+ Temporarily appends GUC settings to the server's postgresql.conf.
+ """
+
+ with open(self._file, "a") as f:
+ print(file=f)
+
+ for n, v in gucs.items():
+ v = str(v)
+
+ # TODO: proper quoting
+ v = v.replace("\\", "\\\\")
+ v = v.replace("'", "\\'")
+ v = "'{}'".format(v)
+
+ print(n, "=", v, file=f)
+
+
+Backup = namedtuple("Backup", "conf, hba")
+
+
+class PostgresServer:
+ """
+ Represents a running PostgreSQL server instance with management utilities.
+ Provides methods for configuration, user/database creation, and server control.
+ """
+
+ def __init__(
+ self,
+ name,
+ bindir,
+ datadir,
+ sockdir,
+ libpq_handle,
+ *,
+ hostaddr: Optional[str] = None,
+ port: Optional[int] = None,
+ ):
+ """
+ Initialize and start a PostgreSQL server instance.
+
+ Args:
+ name: The name of this server instance (for logging purposes)
+ bindir: Path to PostgreSQL bin directory
+ datadir: Path to data directory for this server
+ sockdir: Path to directory for Unix sockets
+ libpq_handle: ctypes handle to libpq
+ hostaddr: If provided, use this specific address (e.g., "127.0.0.2")
+ port: If provided, use this port instead of finding a free one,
+ is currently only allowed if hostaddr is also provided
+ """
+
+ if hostaddr is None and port is not None:
+ raise NotImplementedError("port was provided without hostaddr")
+
+ self.name = name
+ self.datadir = datadir
+ self.sockdir = sockdir
+ self.libpq_handle = libpq_handle
+ self._remaining_timeout_fn: Optional[Callable[[], float]] = None
+ self._bindir = bindir
+ self._pg_ctl = bindir / "pg_ctl"
+ self.log = datadir / "postgresql.log"
+ self._log_start_pos = 0
+
+ # Determine whether to use Unix sockets
+ use_unix_sockets = platform.system() != "Windows" and hostaddr is None
+
+ # Use INITDB_TEMPLATE if available (much faster than running initdb)
+ initdb_template = os.environ.get("INITDB_TEMPLATE")
+ if initdb_template and os.path.isdir(initdb_template):
+ shutil.copytree(initdb_template, datadir)
+ else:
+ if platform.system() == "Windows":
+ auth_method = "trust"
+ else:
+ auth_method = "peer"
+ run(
+ bindir / "initdb",
+ "--no-sync",
+ "--auth",
+ auth_method,
+ "--pgdata",
+ self.datadir,
+ )
+
+ # Figure out a port to listen on. Attempt to reserve both IPv4 and IPv6
+ # addresses in one go.
+ #
+ # Note: socket.has_dualstack_ipv6/create_server are only in Python 3.8+.
+ if hostaddr is not None:
+ # Explicit address provided
+ addrs: list[str] = [hostaddr]
+ temp_sock = socket.socket()
+ if port is None:
+ temp_sock.bind((hostaddr, 0))
+ _, port = temp_sock.getsockname()
+
+ elif hasattr(socket, "has_dualstack_ipv6") and socket.has_dualstack_ipv6():
+ addr = ("::1", 0)
+ temp_sock = socket.create_server(
+ addr, family=socket.AF_INET6, dualstack_ipv6=True
+ )
+
+ hostaddr, port, _, _ = temp_sock.getsockname()
+ assert hostaddr is not None
+ addrs = [hostaddr, "127.0.0.1"]
+
+ else:
+ addr = ("127.0.0.1", 0)
+
+ temp_sock = socket.socket()
+ temp_sock.bind(addr)
+
+ hostaddr, port = temp_sock.getsockname()
+ assert hostaddr is not None
+ addrs = [hostaddr]
+
+ # Store the computed values
+ self.hostaddr = hostaddr
+ self.port = port
+ # Including the host to use for connections - either the socket
+ # directory or TCP address
+ if use_unix_sockets:
+ self.host = str(sockdir)
+ else:
+ self.host = hostaddr
+
+ with open(os.path.join(datadir, "postgresql.conf"), "a") as f:
+ print(file=f)
+ if use_unix_sockets:
+ print(
+ "unix_socket_directories = '{}'".format(sockdir.as_posix()),
+ file=f,
+ )
+ else:
+ # Disable Unix sockets when using TCP to avoid lock conflicts
+ print("unix_socket_directories = ''", file=f)
+ print("listen_addresses = '{}'".format(",".join(addrs)), file=f)
+ print("port =", port, file=f)
+ print("log_connections = all", file=f)
+ print("fsync = off", file=f)
+ print("datestyle = 'ISO'", file=f)
+ print("timezone = 'UTC'", file=f)
+
+ # Between closing of the socket, s, and server start, we're racing
+ # against anything that wants to open up ephemeral ports, so try not to
+ # put any new work here.
+
+ temp_sock.close()
+ self.pg_ctl("start")
+
+ # Read the PID file to get the postmaster PID
+ with open(os.path.join(datadir, "postmaster.pid")) as f:
+ self.pid = int(f.readline().strip())
+
+ # ExitStack for cleanup callbacks
+ self._cleanup_stack = contextlib.ExitStack()
+
+ def current_log_position(self):
+ """Get the current end position of the log file."""
+ if self.log.exists():
+ return self.log.stat().st_size
+ return 0
+
+ def reset_log_position(self):
+ """Mark current log position as start for log_content()."""
+ self._log_start_pos = self.current_log_position()
+
+ @contextlib.contextmanager
+ def start_new_test(self, remaining_timeout):
+ """
+ Prepare server for a new test.
+
+ Sets timeout, resets log position, and enters a cleanup subcontext.
+ """
+ self.set_timeout(remaining_timeout)
+ self.reset_log_position()
+ with self.subcontext():
+ yield self
+
+ def psql(self, *args):
+ """Run psql with the given arguments."""
+ self._run(os.path.join(self._bindir, "psql"), "-w", *args)
+
+ def sql(self, query):
+ """Execute a SQL query via libpq. Returns simplified results."""
+ with self.connect() as conn:
+ return conn.sql(query)
+
+ def pg_ctl(self, *args):
+ """Run pg_ctl with the given arguments."""
+ self._run(self._pg_ctl, "--pgdata", self.datadir, "--log", self.log, *args)
+
+ def _run(self, cmd, *args, addenv: Optional[dict] = None):
+ """Run a command with PG* environment variables set."""
+ subenv = dict(os.environ)
+ subenv.update(
+ {
+ "PGHOST": str(self.host),
+ "PGPORT": str(self.port),
+ "PGDATABASE": "postgres",
+ "PGDATA": str(self.datadir),
+ }
+ )
+ if addenv:
+ subenv.update(addenv)
+ run(cmd, *args, env=subenv)
+
+ def create_users(self, *userkeys: str):
+ """Create test users and register them for cleanup."""
+ usermap = {}
+ for u in userkeys:
+ name = u + "user"
+ usermap[u] = name
+ self.psql("-c", "CREATE USER " + name)
+ self._cleanup_stack.callback(self.psql, "-c", "DROP USER " + name)
+ return usermap
+
+ def create_dbs(self, *dbkeys: str):
+ """Create test databases and register them for cleanup."""
+ dbmap = {}
+ for d in dbkeys:
+ name = d + "db"
+ dbmap[d] = name
+ self.psql("-c", "CREATE DATABASE " + name)
+ self._cleanup_stack.callback(self.psql, "-c", "DROP DATABASE " + name)
+ return dbmap
+
+ @contextlib.contextmanager
+ def reloading(self):
+ """
+ Provides a context manager for making configuration changes.
+
+ If the context suite finishes successfully, the configuration will
+ be reloaded via pg_ctl. On teardown, the configuration changes will
+ be unwound, and the server will be signaled to reload again.
+
+ The context target contains the following attributes which can be
+ used to configure the server:
+ - .conf: modifies postgresql.conf
+ - .hba: modifies pg_hba.conf
+
+ For example:
+
+ with pg_server_session.reloading() as s:
+ s.conf.set(log_connections="on")
+ s.hba.prepend("local all all trust")
+ """
+ # Push a reload onto the stack before making any other
+ # unwindable changes. That way the order of operations will be
+ #
+ # # test
+ # - config change 1
+ # - config change 2
+ # - reload
+ # # teardown
+ # - undo config change 2
+ # - undo config change 1
+ # - reload
+ #
+ self._cleanup_stack.callback(self.pg_ctl, "reload")
+ yield self._backup_configuration()
+
+ # Now actually reload
+ self.pg_ctl("reload")
+
+ @contextlib.contextmanager
+ def restarting(self):
+ """Like .reloading(), but with a full server restart."""
+ self._cleanup_stack.callback(self.pg_ctl, "restart")
+ yield self._backup_configuration()
+ self.pg_ctl("restart")
+
+ def _backup_configuration(self):
+ # Wrap the existing HBA and configuration with FileBackups.
+ return Backup(
+ hba=self._cleanup_stack.enter_context(HBA(self.datadir)),
+ conf=self._cleanup_stack.enter_context(Config(self.datadir)),
+ )
+
+ @contextlib.contextmanager
+ def subcontext(self):
+ """
+ Create a new cleanup context for per-test isolation.
+
+ Temporarily replaces the cleanup stack so that any cleanup callbacks
+ registered within this context will be cleaned up when the context exits.
+ """
+ old_stack = self._cleanup_stack
+ self._cleanup_stack = contextlib.ExitStack()
+ try:
+ self._cleanup_stack.__enter__()
+ yield self
+ finally:
+ self._cleanup_stack.__exit__(None, None, None)
+ self._cleanup_stack = old_stack
+
+ def stop(self, mode="fast"):
+ """
+ Stop the PostgreSQL server instance.
+
+ Ignores failures if the server is already stopped.
+ """
+ try:
+ self.pg_ctl("stop", "--mode", mode)
+ except subprocess.CalledProcessError:
+ # Server may have already been stopped
+ pass
+
+ def log_content(self) -> str:
+ """Return log content from the current context's start position."""
+ with open(self.log) as f:
+ f.seek(self._log_start_pos)
+ return f.read()
+
+ @contextlib.contextmanager
+ def log_contains(self, pattern, times=None):
+ """
+ Context manager that checks if the log matches pattern during the block.
+
+ Args:
+ pattern: The regex pattern to search for.
+ times: If None, any number of matches is accepted.
+ If a number, exactly that many matches are required.
+ """
+ start_pos = self.current_log_position()
+ yield
+ with open(self.log) as f:
+ f.seek(start_pos)
+ content = f.read()
+ if times is None:
+ assert re.search(pattern, content), f"Pattern {pattern!r} not found in log"
+ else:
+ match_count = len(re.findall(pattern, content))
+ assert match_count == times, (
+ f"Expected {times} matches of {pattern!r}, found {match_count}"
+ )
+
+ def cleanup(self):
+ """Run all registered cleanup callbacks."""
+ self._cleanup_stack.close()
+
+ def set_timeout(self, remaining_timeout_fn: Callable[[], float]) -> None:
+ """
+ Set the timeout function for connections.
+ This is typically called by pg fixture for each test.
+ """
+ self._remaining_timeout_fn = remaining_timeout_fn
+
+ def connect(self, **opts) -> PGconn:
+ """
+ Creates a connection to this PostgreSQL server instance.
+
+ Args:
+ **opts: Additional connection options (can override defaults)
+
+ Returns:
+ PGconn: Connected database connection
+
+ Example:
+ conn = pg.connect()
+ conn = pg.connect(dbname='mydb')
+ """
+ if self._remaining_timeout_fn is None:
+ raise RuntimeError(
+ "Timeout function not set. Use set_timeout() or pg fixture."
+ )
+
+ defaults = {
+ "host": self.host,
+ "port": self.port,
+ "dbname": "postgres",
+ }
+ defaults.update(opts)
+
+ return libpq_connect(
+ self.libpq_handle,
+ self._cleanup_stack,
+ self._remaining_timeout_fn,
+ **defaults,
+ )
diff --git a/src/test/pytest/pypg/util.py b/src/test/pytest/pypg/util.py
new file mode 100644
index 00000000000..b2a1e627e4b
--- /dev/null
+++ b/src/test/pytest/pypg/util.py
@@ -0,0 +1,42 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import shlex
+import subprocess
+import sys
+
+
+def eprint(*args, **kwargs):
+ """eprint prints to stderr"""
+ print(*args, file=sys.stderr, **kwargs)
+
+
+def run(*command, check=True, shell=None, silent=False, **kwargs):
+ """run runs the given command and prints it to stderr"""
+
+ if shell is None:
+ shell = len(command) == 1 and isinstance(command[0], str)
+
+ if shell:
+ command = command[0]
+ else:
+ command = list(map(str, command))
+
+ if not silent:
+ if shell:
+ eprint(f"+ {command}")
+ else:
+ # We could normally use shlex.join here, but it's not available in
+ # Python 3.6 which we still like to support
+ unsafe_string_cmd = " ".join(map(shlex.quote, command))
+ eprint(f"+ {unsafe_string_cmd}")
+
+ if silent:
+ kwargs.setdefault("stdout", subprocess.DEVNULL)
+
+ return subprocess.run(command, check=check, shell=shell, **kwargs)
+
+
+def capture(command, *args, stdout=subprocess.PIPE, encoding="utf-8", **kwargs):
+ return run(
+ command, *args, stdout=stdout, encoding=encoding, **kwargs
+ ).stdout.removesuffix("\n")
diff --git a/src/test/pytest/pyt/conftest.py b/src/test/pytest/pyt/conftest.py
new file mode 100644
index 00000000000..dd73917c68c
--- /dev/null
+++ b/src/test/pytest/pyt/conftest.py
@@ -0,0 +1 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
diff --git a/src/test/pytest/pyt/test_errors.py b/src/test/pytest/pyt/test_errors.py
new file mode 100644
index 00000000000..ad109039668
--- /dev/null
+++ b/src/test/pytest/pyt/test_errors.py
@@ -0,0 +1,34 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Tests for libpq error types and SQLSTATE-based exception mapping.
+"""
+
+import pytest
+import libpq
+
+
+def test_syntax_error(conn):
+ """Invalid SQL syntax raises SyntaxError with correct SQLSTATE."""
+ with pytest.raises(libpq.errors.SyntaxError) as exc_info:
+ conn.sql("SELEC 1")
+
+ err = exc_info.value
+ assert err.sqlstate == "42601"
+ assert err.sqlstate_class == "42"
+ assert "syntax" in str(err).lower()
+
+
+def test_unique_violation(conn):
+ """Unique violation includes all error fields and can be caught as parent class."""
+ conn.sql("CREATE TEMP TABLE test_uv (id int CONSTRAINT test_uv_pk PRIMARY KEY)")
+ conn.sql("INSERT INTO test_uv VALUES (1)")
+
+ with pytest.raises(libpq.errors.UniqueViolation) as exc_info:
+ conn.sql("INSERT INTO test_uv VALUES (1)")
+
+ err = exc_info.value
+ assert err.sqlstate == "23505"
+ assert err.table_name == "test_uv"
+ assert err.constraint_name == "test_uv_pk"
+ assert err.detail == "Key (id)=(1) already exists."
diff --git a/src/test/pytest/pyt/test_libpq.py b/src/test/pytest/pyt/test_libpq.py
new file mode 100644
index 00000000000..4fcf4056f41
--- /dev/null
+++ b/src/test/pytest/pyt/test_libpq.py
@@ -0,0 +1,172 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import contextlib
+import os
+import socket
+import struct
+import threading
+from typing import Callable
+
+import pytest
+
+from libpq import connstr, LibpqError
+
+
[email protected](
+ "opts, expected",
+ [
+ (dict(), ""),
+ (dict(port=5432), "port=5432"),
+ (dict(port=5432, dbname="postgres"), "port=5432 dbname=postgres"),
+ (dict(host=""), "host=''"),
+ (dict(host=" "), r"host=' '"),
+ (dict(keyword="'"), r"keyword=\'"),
+ (dict(keyword=" \\' "), r"keyword=' \\\' '"),
+ ],
+)
+def test_connstr(opts, expected):
+ """Tests the escape behavior for connstr()."""
+ assert connstr(opts) == expected
+
+
+def test_must_connect_errors(connect):
+ """Tests that connect() raises LibpqError."""
+ with pytest.raises(LibpqError, match="invalid connection option"):
+ connect(some_unknown_keyword="whatever")
+
+
[email protected]
+def local_server(tmp_path, remaining_timeout):
+ """
+ Opens up a local UNIX socket for mocking a Postgres server on a background
+ thread. See the _Server API for usage.
+
+ This fixture requires AF_UNIX support; dependent tests will be skipped on
+ platforms that don't provide it.
+ """
+
+ try:
+ from socket import AF_UNIX
+ except ImportError:
+ pytest.skip("AF_UNIX not supported on this platform")
+
+ class _Server(contextlib.ExitStack):
+ """
+ Implementation class for local_server. See .background() for the primary
+ entry point for tests. Postgres clients may connect to this server via
+ local_server.host/local_server.port.
+
+ _Server derives from contextlib.ExitStack to provide easy cleanup of
+ associated resources; see the documentation for that class for a full
+ explanation.
+ """
+
+ def __init__(self):
+ super().__init__()
+
+ self.host = tmp_path
+ self.port = 5432
+
+ self._thread = None
+ self._thread_exc = None
+ self._listener = self.enter_context(
+ socket.socket(AF_UNIX, socket.SOCK_STREAM),
+ )
+
+ def bind_and_listen(self):
+ """
+ Does the actual work of binding the UNIX socket using the Postgres
+ server conventions and listening for connections.
+
+ The listen backlog is currently hardcoded to one.
+ """
+ sockfile = self.host / ".s.PGSQL.{}".format(self.port)
+
+ # Lock down the permissions on the new socket.
+ prev_mask = os.umask(0o077)
+
+ # Bind (creating the socket file), and immediately register it for
+ # deletion from disk when the stack is cleaned up.
+ self._listener.bind(bytes(sockfile))
+ self.callback(os.unlink, sockfile)
+
+ os.umask(prev_mask)
+
+ self._listener.listen(1)
+
+ def background(self, fn: Callable[[socket.socket], None]) -> None:
+ """
+ Accepts a client connection on a background thread and passes it to
+ the provided callback. Any exceptions raised from the callback will
+ be re-raised on the main thread during fixture teardown.
+
+ Blocking operations on the connected socket default to using the
+ remaining_timeout(), though this can be changed by the test via the
+ socket's .settimeout().
+ """
+
+ def _bg():
+ try:
+ self._listener.settimeout(remaining_timeout())
+ sock, _ = self._listener.accept()
+
+ with sock:
+ sock.settimeout(remaining_timeout())
+ fn(sock)
+
+ except Exception as e:
+ # Save the exception for re-raising on the main thread.
+ self._thread_exc = e
+
+ # TODO: rather than using callback(), consider explicitly signaling
+ # the fn() implementation to stop early if we get an exception.
+ # Otherwise we'll hang until the end of the timeout.
+ self._thread = threading.Thread(target=_bg)
+ self.callback(self._join)
+
+ self._thread.start()
+
+ def _join(self):
+ """
+ Waits for the background thread to finish and raises any thrown
+ exception. This is called during fixture teardown.
+ """
+ # Give a little bit of wiggle room on the join timeout, since we're
+ # racing against the test's own use of remaining_timeout(). (It's
+ # preferable to let tests report timeouts; the stack traces will
+ # help with debugging.)
+ self._thread.join(remaining_timeout() + 1)
+ if self._thread.is_alive():
+ raise TimeoutError("background thread is still running after timeout")
+
+ if self._thread_exc is not None:
+ raise self._thread_exc
+
+ with _Server() as s:
+ s.bind_and_listen()
+ yield s
+
+
+def test_connection_is_finished_on_error(connect, local_server):
+ """Tests that PQfinish() gets called at the end of testing."""
+ expected_error = "something is wrong"
+
+ def serve_error(s: socket.socket) -> None:
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+
+ # Quick check for the startup packet version.
+ version = struct.unpack("!HH", s.recv(4))
+ assert version == (3, 0)
+
+ # Discard the remainder of the startup packet and send a v2 error.
+ s.recv(pktlen - 8)
+ s.send(b"E" + expected_error.encode() + b"\0")
+
+ # And now the socket should be closed.
+ assert not s.recv(1), "client sent unexpected data"
+
+ local_server.background(serve_error)
+
+ with pytest.raises(LibpqError, match=expected_error):
+ # Exiting this context should result in PQfinish().
+ connect(host=local_server.host, port=local_server.port)
diff --git a/src/test/pytest/pyt/test_multi_server.py b/src/test/pytest/pyt/test_multi_server.py
new file mode 100644
index 00000000000..8ee045b0cc8
--- /dev/null
+++ b/src/test/pytest/pyt/test_multi_server.py
@@ -0,0 +1,46 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Tests demonstrating multi-server functionality using create_pg fixture.
+
+These tests verify that the pytest infrastructure correctly handles
+multiple PostgreSQL server instances within a single test, and that
+module-scoped servers persist across tests.
+"""
+
+import pytest
+
+
+def test_multiple_servers_basic(create_pg):
+ """Test that we can create and connect to multiple servers."""
+ node1 = create_pg("primary")
+ node2 = create_pg("secondary")
+
+ conn1 = node1.connect()
+ conn2 = node2.connect()
+
+ # Each server should have its own data directory
+ datadir1 = conn1.sql("SHOW data_directory")
+ datadir2 = conn2.sql("SHOW data_directory")
+ assert datadir1 != datadir2
+
+ # Each server should be listening on a different port
+ assert node1.port != node2.port
+
+
[email protected](scope="module")
+def shared_server(create_pg_module):
+ """A server shared across all tests in this module."""
+ server = create_pg_module("shared")
+ server.sql("CREATE TABLE module_state (value int DEFAULT 0)")
+ return server
+
+
+def test_module_server_create_row(shared_server):
+ """First test: create a row in the shared server."""
+ shared_server.connect().sql("INSERT INTO module_state VALUES (42)")
+
+
+def test_module_server_see_row(shared_server):
+ """Second test: verify we see the row from the previous test."""
+ assert shared_server.connect().sql("SELECT value FROM module_state") == 42
diff --git a/src/test/pytest/pyt/test_query_helpers.py b/src/test/pytest/pyt/test_query_helpers.py
new file mode 100644
index 00000000000..abcd9084214
--- /dev/null
+++ b/src/test/pytest/pyt/test_query_helpers.py
@@ -0,0 +1,347 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Tests for query helper functions with type conversion and result simplification.
+"""
+
+import uuid
+
+import pytest
+
+
+def test_single_cell_int(conn):
+ """Single cell integer query returns just the value."""
+ result = conn.sql("SELECT 1")
+ assert result == 1
+ assert isinstance(result, int)
+
+
+def test_single_cell_string(conn):
+ """Single cell string query returns just the value."""
+ result = conn.sql("SELECT 'hello'")
+ assert result == "hello"
+ assert isinstance(result, str)
+
+
+def test_single_cell_bool(conn):
+ """Single cell boolean query returns just the value."""
+
+ result = conn.sql("SELECT true")
+ assert result is True
+ assert isinstance(result, bool)
+
+ result = conn.sql("SELECT false")
+ assert result is False
+
+
+def test_single_cell_float(conn):
+ """Single cell float query returns just the value."""
+
+ result = conn.sql("SELECT 3.14::float4")
+ assert isinstance(result, float)
+ assert abs(result - 3.14) < 0.01
+
+
+def test_single_cell_null(conn):
+ """Single cell NULL query returns None."""
+
+ result = conn.sql("SELECT NULL")
+ assert result is None
+
+
+def test_single_row_multiple_columns(conn):
+ """Single row with multiple columns returns a tuple."""
+
+ result = conn.sql("SELECT 1, 'hello', true")
+ assert result == (1, "hello", True)
+ assert isinstance(result, tuple)
+
+
+def test_single_column_multiple_rows(conn):
+ """Single column with multiple rows returns a list of values."""
+
+ result = conn.sql("SELECT * FROM generate_series(1, 3)")
+ assert result == [1, 2, 3]
+ assert isinstance(result, list)
+
+
+def test_multiple_rows_and_columns(conn):
+ """Multiple rows and columns returns list of tuples."""
+
+ result = conn.sql("SELECT * FROM (VALUES (1, 'a'), (2, 'b'), (3, 'c')) AS t")
+ assert result == [(1, "a"), (2, "b"), (3, "c")]
+ assert isinstance(result, list)
+ assert all(isinstance(row, tuple) for row in result)
+
+
+def test_empty_result(conn):
+ """Empty result set returns empty list."""
+
+ result = conn.sql("SELECT 1 WHERE false")
+ assert result == []
+
+
+def test_query_error_handling(conn):
+ """Query errors raise RuntimeError with actual error message."""
+
+ with pytest.raises(RuntimeError) as exc_info:
+ conn.sql("SELECT * FROM nonexistent_table")
+
+ error_msg = str(exc_info.value)
+ assert "nonexistent_table" in error_msg or "does not exist" in error_msg
+
+
+def test_division_by_zero_error(conn):
+ """Division by zero raises RuntimeError."""
+
+ with pytest.raises(RuntimeError) as exc_info:
+ conn.sql("SELECT 1/0")
+
+ error_msg = str(exc_info.value)
+ assert "division by zero" in error_msg.lower()
+
+
+def test_simple_exec_create_table(conn):
+ """sql for CREATE TABLE returns None."""
+
+ result = conn.sql("CREATE TEMP TABLE test_table (id int, name text)")
+ assert result is None
+
+ # Verify table was created
+ count = conn.sql("SELECT COUNT(*) FROM test_table")
+ assert count == 0
+
+
+def test_simple_exec_insert(conn):
+ """sql for INSERT returns None."""
+
+ conn.sql("CREATE TEMP TABLE test_table (id int, name text)")
+ result = conn.sql("INSERT INTO test_table VALUES (1, 'Alice'), (2, 'Bob')")
+ assert result is None
+
+ # Verify data was inserted
+ count = conn.sql("SELECT COUNT(*) FROM test_table")
+ assert count == 2
+
+
+def test_type_conversion_mixed(conn):
+ """Test mixed type conversion in a single row."""
+
+ result = conn.sql("SELECT 42::int4, 123::int8, 3.14::float8, 'text', true, NULL")
+ assert result == (42, 123, 3.14, "text", True, None)
+ assert isinstance(result[0], int)
+ assert isinstance(result[1], int)
+ assert isinstance(result[2], float)
+ assert isinstance(result[3], str)
+ assert isinstance(result[4], bool)
+ assert result[5] is None
+
+
+def test_multiple_queries_same_connection(conn):
+ """Test running multiple queries on the same connection."""
+
+ result1 = conn.sql("SELECT 1")
+ assert result1 == 1
+
+ result2 = conn.sql("SELECT 'hello', 'world'")
+ assert result2 == ("hello", "world")
+
+ result3 = conn.sql("SELECT * FROM generate_series(1, 5)")
+ assert result3 == [1, 2, 3, 4, 5]
+
+
+def test_date_type(conn):
+ """Test date type conversion."""
+ import datetime
+
+ result = conn.sql("SELECT '2025-10-20'::date")
+ assert result == datetime.date(2025, 10, 20)
+ assert isinstance(result, datetime.date)
+
+
+def test_timestamp_type(conn):
+ """Test timestamp type conversion."""
+ import datetime
+
+ result = conn.sql("SELECT '2025-10-20 15:30:45'::timestamp")
+ assert result == datetime.datetime(2025, 10, 20, 15, 30, 45)
+ assert isinstance(result, datetime.datetime)
+
+
+def test_time_type(conn):
+ """Test time type conversion."""
+ import datetime
+
+ result = conn.sql("SELECT '15:30:45'::time")
+ assert result == datetime.time(15, 30, 45)
+ assert isinstance(result, datetime.time)
+
+
+def test_numeric_type(conn):
+ """Test numeric/decimal type conversion."""
+ import decimal
+
+ result = conn.sql("SELECT 123.456::numeric")
+ assert result == decimal.Decimal("123.456")
+ assert isinstance(result, decimal.Decimal)
+
+
+def test_int_array(conn):
+ """Test integer array type conversion."""
+
+ result = conn.sql("SELECT ARRAY[1, 2, 3, 4, 5]")
+ assert result == [1, 2, 3, 4, 5]
+ assert isinstance(result, list)
+ assert all(isinstance(x, int) for x in result)
+
+
+def test_text_array(conn):
+ """Test text array type conversion."""
+
+ result = conn.sql("SELECT ARRAY['hello', 'world', 'test']")
+ assert result == ["hello", "world", "test"]
+ assert isinstance(result, list)
+ assert all(isinstance(x, str) for x in result)
+
+
+def test_bool_array(conn):
+ """Test boolean array type conversion."""
+
+ result = conn.sql("SELECT ARRAY[true, false, true]")
+ assert result == [True, False, True]
+ assert isinstance(result, list)
+ assert all(isinstance(x, bool) for x in result)
+
+
+def test_empty_array(conn):
+ """Test empty array type conversion."""
+
+ result = conn.sql("SELECT ARRAY[]::int[]")
+ assert result == []
+ assert isinstance(result, list)
+
+
+def test_json_type(conn):
+ """Test JSON type (parsed to dict)."""
+
+ result = conn.sql('SELECT \'{"key": "value"}\'::json')
+ assert isinstance(result, dict)
+ assert result == {"key": "value"}
+
+
+def test_jsonb_type(conn):
+ """Test JSONB type (parsed to dict)."""
+
+ result = conn.sql('SELECT \'{"name": "test", "count": 42}\'::jsonb')
+ assert isinstance(result, dict)
+ assert result == {"name": "test", "count": 42}
+
+
+def test_json_array(conn):
+ """Test JSON array type."""
+
+ result = conn.sql("SELECT '[1, 2, 3, 4, 5]'::json")
+ assert isinstance(result, list)
+ assert result == [1, 2, 3, 4, 5]
+
+
+def test_json_nested(conn):
+ """Test nested JSON object."""
+
+ result = conn.sql(
+ 'SELECT \'{"user": {"id": 1, "name": "Alice"}, "active": true}\'::json'
+ )
+ assert isinstance(result, dict)
+ assert result == {"user": {"id": 1, "name": "Alice"}, "active": True}
+
+
+def test_mixed_types_with_arrays(conn):
+ """Test mixed types including arrays in a single row."""
+
+ result = conn.sql("SELECT 42, 'text', ARRAY[1, 2, 3], true")
+ assert result == (42, "text", [1, 2, 3], True)
+ assert isinstance(result[0], int)
+ assert isinstance(result[1], str)
+ assert isinstance(result[2], list)
+ assert isinstance(result[3], bool)
+
+
+def test_uuid_type(conn):
+ """Test UUID type conversion."""
+ test_uuid = "550e8400-e29b-41d4-a716-446655440000"
+ result = conn.sql(f"SELECT '{test_uuid}'::uuid")
+ assert result == uuid.UUID(test_uuid)
+ assert isinstance(result, uuid.UUID)
+
+
+def test_uuid_generation(conn):
+ """Test generated UUID type conversion."""
+ result = conn.sql("SELECT uuidv4()")
+ assert isinstance(result, uuid.UUID)
+ # Check it's a valid UUID by ensuring it can be converted to string
+ assert len(str(result)) == 36 # UUID string format length
+
+
+def test_text_array_with_commas(conn):
+ """Test text array with elements containing commas."""
+
+ result = conn.sql("SELECT ARRAY['A,B', 'C', ' D ']")
+ assert result == ["A,B", "C", " D "]
+
+
+def test_text_array_with_quotes(conn):
+ """Test text array with elements containing quotes."""
+
+ result = conn.sql(r"SELECT ARRAY[E'a\"b', 'c']")
+ assert result == ['a"b', "c"]
+
+
+def test_text_array_with_backslash(conn):
+ """Test text array with elements containing backslashes."""
+
+ result = conn.sql(r"SELECT ARRAY[E'a\\b', 'c']")
+ assert result == ["a\\b", "c"]
+
+
+def test_json_array_type(conn):
+ """Test array of JSON values with embedded quotes and commas."""
+
+ result = conn.sql("""SELECT ARRAY['{"abc": 123, "xyz": 456}'::json]""")
+ assert result == [{"abc": 123, "xyz": 456}]
+
+
+def test_json_array_multiple(conn):
+ """Test array of multiple JSON objects."""
+
+ result = conn.sql(
+ """SELECT ARRAY['{"a": 1}'::json, '{"b": 2}'::json, '["x", "y"]'::json]"""
+ )
+ assert result == [{"a": 1}, {"b": 2}, ["x", "y"]]
+
+
+def test_2d_int_array(conn):
+ """Test 2D integer array."""
+
+ result = conn.sql("SELECT ARRAY[[1,2],[3,4]]")
+ assert result == [[1, 2], [3, 4]]
+
+
+def test_2d_text_array(conn):
+ """Test 2D integer array."""
+
+ result = conn.sql("SELECT ARRAY[['a','b'],['c','d,e']]")
+ assert result == [["a", "b"], ["c", "d,e"]]
+
+
+def test_3d_int_array(conn):
+ """Test 3D integer array."""
+
+ result = conn.sql("SELECT ARRAY[[[1,2],[3,4]],[[5,6],[7,8]]]")
+ assert result == [[[1, 2], [3, 4]], [[5, 6], [7, 8]]]
+
+
+def test_array_with_null(conn):
+ """Test array with NULL elements."""
+
+ result = conn.sql("SELECT ARRAY[1, NULL, 3]")
+ assert result == [1, None, 3]
diff --git a/src/tools/generate_pytest_libpq_errors.py b/src/tools/generate_pytest_libpq_errors.py
new file mode 100755
index 00000000000..ba92891c17a
--- /dev/null
+++ b/src/tools/generate_pytest_libpq_errors.py
@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Generate src/test/pytest/libpq/_generated_errors.py from errcodes.txt.
+"""
+
+import sys
+from pathlib import Path
+
+
+ACRONYMS = {"sql", "fdw"}
+WORD_MAP = {
+ "sqlclient": "SQLClient",
+ "sqlserver": "SQLServer",
+ "sqlconnection": "SQLConnection",
+}
+
+
+def snake_to_pascal(name: str) -> str:
+ """Convert snake_case to PascalCase, keeping acronyms uppercase."""
+ words = []
+ for word in name.split("_"):
+ if word in WORD_MAP:
+ words.append(WORD_MAP[word])
+ elif word in ACRONYMS:
+ words.append(word.upper())
+ else:
+ words.append(word.capitalize())
+ return "".join(words)
+
+
+def parse_errcodes(path: Path):
+ """Parse errcodes.txt and return list of (sqlstate, macro_name, spec_name) tuples."""
+ errors = []
+
+ with open(path) as f:
+ for line in f:
+ parts = line.split()
+ if len(parts) >= 4 and len(parts[0]) == 5:
+ sqlstate, _, macro_name, spec_name = parts[:4]
+ errors.append((sqlstate, macro_name, spec_name))
+
+ return errors
+
+
+def macro_to_class_name(macro_name: str) -> str:
+ """Convert ERRCODE_FOO_BAR to FooBar."""
+ name = macro_name.removeprefix("ERRCODE_")
+ # Move WARNING prefix to the end as a suffix
+ if name.startswith("WARNING_"):
+ name = name.removeprefix("WARNING_") + "_WARNING"
+ return snake_to_pascal(name.lower())
+
+
+def generate_errors(errcodes_path: Path):
+ """Generate the _generated_errors.py content."""
+ errors = parse_errcodes(errcodes_path)
+
+ # Find spec_names that appear more than once (collisions)
+ spec_name_counts: dict[str, int] = {}
+ for _, _, spec_name in errors:
+ spec_name_counts[spec_name] = spec_name_counts.get(spec_name, 0) + 1
+ colliding_spec_names = {
+ name for name, count in spec_name_counts.items() if count > 1
+ }
+
+ lines = [
+ "# Copyright (c) 2025, PostgreSQL Global Development Group",
+ "# This file is generated by src/tools/generate_pytest_libpq_errors.py - do not edit directly.",
+ "",
+ '"""',
+ "Generated PostgreSQL error classes mapped from SQLSTATE codes.",
+ '"""',
+ "",
+ "from typing import Dict",
+ "",
+ "from ._error_base import LibpqError, LibpqWarning",
+ "",
+ "",
+ ]
+
+ generated_classes = {"LibpqError"}
+ sqlstate_to_exception = {}
+
+ for sqlstate, macro_name, spec_name in errors:
+ # 000 errors define the parent class for all errors in this SQLSTATE class
+ if sqlstate.endswith("000"):
+ exc_name = snake_to_pascal(spec_name)
+ if exc_name == "Warning":
+ parent = "LibpqWarning"
+ else:
+ parent = "LibpqError"
+ else:
+ if spec_name in colliding_spec_names:
+ exc_name = macro_to_class_name(macro_name)
+ else:
+ exc_name = snake_to_pascal(spec_name)
+ # Use parent class if available, otherwise LibpqError
+ parent = sqlstate_to_exception.get(sqlstate[:2] + "000", "LibpqError")
+ # Warnings should end with "Warning"
+ if parent == "Warning" and not exc_name.endswith("Warning"):
+ exc_name += "Warning"
+
+ generated_classes.add(exc_name)
+ sqlstate_to_exception[sqlstate] = exc_name
+ lines.extend(
+ [
+ f"class {exc_name}({parent}):",
+ f' """SQLSTATE {sqlstate} - {spec_name.replace("_", " ")}."""',
+ "",
+ " pass",
+ "",
+ "",
+ ]
+ )
+
+ lines.append("SQLSTATE_TO_EXCEPTION: Dict[str, type] = {")
+ for sqlstate, exc_name in sqlstate_to_exception.items():
+ lines.append(f' "{sqlstate}": {exc_name},')
+ lines.extend(["}", "", ""])
+
+ all_exports = list(generated_classes) + ["SQLSTATE_TO_EXCEPTION"]
+ lines.append("__all__ = [")
+ for name in all_exports:
+ lines.append(f' "{name}",')
+ lines.append("]")
+
+ return "\n".join(lines) + "\n"
+
+
+if __name__ == "__main__":
+ script_dir = Path(__file__).resolve().parent
+ src_root = script_dir.parent.parent
+
+ errcodes_path = src_root / "src" / "backend" / "utils" / "errcodes.txt"
+ output_path = (
+ src_root / "src" / "test" / "pytest" / "libpq" / "_generated_errors.py"
+ )
+
+ if not errcodes_path.exists():
+ print(f"Error: {errcodes_path} not found", file=sys.stderr)
+ sys.exit(1)
+
+ output = generate_errors(errcodes_path)
+ output_path.write_text(output)
+ print(f"Generated {output_path}")
--
2.52.0
[text/x-patch] v5-0005-Convert-load-balance-tests-from-perl-to-python.patch (16.9K, 6-v5-0005-Convert-load-balance-tests-from-perl-to-python.patch)
download | inline diff:
From 92b671c822f6f68247fc864ec1dbe03484935bd7 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Dec 2025 12:31:43 +0100
Subject: [PATCH v5 5/7] Convert load balance tests from perl to python
---
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/meson.build | 7 +-
src/interfaces/libpq/pyt/test_load_balance.py | 170 ++++++++++++++++++
.../libpq/t/003_load_balance_host_list.pl | 94 ----------
.../libpq/t/004_load_balance_dns.pl | 144 ---------------
5 files changed, 176 insertions(+), 240 deletions(-)
create mode 100644 src/interfaces/libpq/pyt/test_load_balance.py
delete mode 100644 src/interfaces/libpq/t/003_load_balance_host_list.pl
delete mode 100644 src/interfaces/libpq/t/004_load_balance_dns.pl
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 9fe321147fc..41ea88c7388 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -167,6 +167,7 @@ check installcheck: export PATH := $(CURDIR)/test:$(PATH)
check: test-build all
$(prove_check)
+ $(pytest_check)
installcheck: test-build all
$(prove_installcheck)
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index b259c998fa2..6d62ac17edb 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -150,8 +150,6 @@ tests += {
'tests': [
't/001_uri.pl',
't/002_api.pl',
- 't/003_load_balance_host_list.pl',
- 't/004_load_balance_dns.pl',
't/005_negotiate_encryption.pl',
't/006_service.pl',
],
@@ -162,6 +160,11 @@ tests += {
},
'deps': libpq_test_deps,
},
+ 'pytest': {
+ 'tests': [
+ 'pyt/test_load_balance.py',
+ ],
+ },
}
subdir('po', if_found: libintl)
diff --git a/src/interfaces/libpq/pyt/test_load_balance.py b/src/interfaces/libpq/pyt/test_load_balance.py
new file mode 100644
index 00000000000..0af46d8f37d
--- /dev/null
+++ b/src/interfaces/libpq/pyt/test_load_balance.py
@@ -0,0 +1,170 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+"""
+Tests for load_balance_hosts connection parameter.
+
+These tests verify that libpq correctly handles load balancing across multiple
+PostgreSQL servers specified in the connection string.
+"""
+
+import platform
+import re
+
+import pytest
+
+from libpq import LibpqError
+import pypg
+
+
[email protected](scope="module")
+def load_balance_nodes_hostlist(create_pg_module):
+ """
+ Create 3 PostgreSQL nodes with different socket directories.
+
+ Each node has its own Unix socket directory for isolation.
+ Returns a tuple of (nodes, connect).
+ """
+ nodes = [create_pg_module() for _ in range(3)]
+
+ hostlist = ",".join(node.host for node in nodes)
+ portlist = ",".join(str(node.port) for node in nodes)
+
+ def connect(**kwargs):
+ return nodes[0].connect(host=hostlist, port=portlist, **kwargs)
+
+ return nodes, connect
+
+
[email protected](scope="module")
+def load_balance_nodes_dns(create_pg_module):
+ """
+ Create 3 PostgreSQL nodes on the same port but different IP addresses.
+
+ Uses 127.0.0.1, 127.0.0.2, 127.0.0.3 with a shared port, so that
+ connections to 'pg-loadbalancetest' can be load balanced via DNS.
+
+ Since setting up a DNS server is more effort than we consider reasonable to
+ run this test, this situation is instead imitated by using a hosts file
+ where a single hostname maps to multiple different IP addresses. This test
+ requires the administrator to add the following lines to the hosts file (if
+ we detect that this hasn't happened we skip the test):
+
+ 127.0.0.1 pg-loadbalancetest
+ 127.0.0.2 pg-loadbalancetest
+ 127.0.0.3 pg-loadbalancetest
+
+ Windows or Linux are required to run this test because these OSes allow
+ binding to 127.0.0.2 and 127.0.0.3 addresses by default, but other OSes
+ don't. We need to bind to different IP addresses, so that we can use these
+ different IP addresses in the hosts file.
+
+ The hosts file needs to be prepared before running this test. We don't do
+ it on the fly, because it requires root permissions to change the hosts
+ file. In CI we set up the previously mentioned rules in the hosts file, so
+ that this load balancing method is tested.
+
+ Requires PG_TEST_EXTRA=load_balance because it requires this manual hosts
+ file configuration and also uses TCP with trust auth, which is potentially
+ unsafe on multiuser systems.
+ """
+ pypg.skip_unless_test_extras("load_balance")
+
+ if platform.system() not in ("Linux", "Windows"):
+ pytest.skip("DNS load balance test only supported on Linux and Windows")
+
+ if platform.system() == "Windows":
+ hosts_path = r"c:\Windows\System32\Drivers\etc\hosts"
+ else:
+ hosts_path = "/etc/hosts"
+
+ try:
+ with open(hosts_path) as f:
+ hosts_content = f.read()
+ except (OSError, IOError):
+ pytest.skip(f"Could not read hosts file: {hosts_path}")
+
+ count = len(re.findall(r"127\.0\.0\.[1-3]\s+pg-loadbalancetest", hosts_content))
+ if count != 3:
+ pytest.skip("hosts file not prepared for DNS load balance test")
+
+ first_node = create_pg_module(hostaddr="127.0.0.1")
+ nodes = [
+ first_node,
+ create_pg_module(hostaddr="127.0.0.2", port=first_node.port),
+ create_pg_module(hostaddr="127.0.0.3", port=first_node.port),
+ ]
+
+ # Allow trust authentication for TCP connections from loopback
+ for node in nodes:
+ hba_path = node.datadir / "pg_hba.conf"
+ with open(hba_path, "r") as f:
+ original_content = f.read()
+ with open(hba_path, "w") as f:
+ f.write("host all all 127.0.0.0/8 trust\n")
+ f.write(original_content)
+ node.pg_ctl("reload")
+
+ def connect(**kwargs):
+ return nodes[0].connect(host="pg-loadbalancetest", **kwargs)
+
+ return nodes, connect
+
+
[email protected](scope="module", params=["hostlist", "dns"])
+def load_balance_nodes(request):
+ """
+ Parametrized fixture providing both load balancing test environments.
+ """
+ return request.getfixturevalue(f"load_balance_nodes_{request.param}")
+
+
+def test_load_balance_hosts_invalid_value(load_balance_nodes):
+ """load_balance_hosts doesn't accept unknown values."""
+ _, connect = load_balance_nodes
+
+ with pytest.raises(
+ LibpqError, match='invalid load_balance_hosts value: "doesnotexist"'
+ ):
+ connect(load_balance_hosts="doesnotexist")
+
+
+def test_load_balance_hosts_disable(load_balance_nodes):
+ """load_balance_hosts=disable always connects to the first node."""
+ nodes, connect = load_balance_nodes
+
+ with nodes[0].log_contains("connection received"):
+ connect(load_balance_hosts="disable")
+
+
+def test_load_balance_hosts_random_distribution(load_balance_nodes):
+ """load_balance_hosts=random distributes connections across all nodes."""
+ nodes, connect = load_balance_nodes
+
+ for _ in range(50):
+ connect(load_balance_hosts="random")
+
+ occurrences = [
+ len(re.findall("connection received", node.log_content())) for node in nodes
+ ]
+
+ # Statistically, each node should receive at least one connection.
+ # The probability of any node receiving 0 connections is (2/3)^50 ≈ 1.57e-9
+ assert occurrences[0] > 0, "node1 should receive at least one connection"
+ assert occurrences[1] > 0, "node2 should receive at least one connection"
+ assert occurrences[2] > 0, "node3 should receive at least one connection"
+ assert sum(occurrences) == 50, "total connections should be 50"
+
+
+def test_load_balance_hosts_failover(load_balance_nodes):
+ """load_balance_hosts continues trying hosts until it finds a working one."""
+ nodes, connect = load_balance_nodes
+
+ nodes[0].stop()
+ nodes[1].stop()
+
+ with nodes[2].log_contains("connection received"):
+ connect(load_balance_hosts="disable")
+
+ with nodes[2].log_contains("connection received", times=5):
+ for _ in range(5):
+ connect(load_balance_hosts="random")
diff --git a/src/interfaces/libpq/t/003_load_balance_host_list.pl b/src/interfaces/libpq/t/003_load_balance_host_list.pl
deleted file mode 100644
index 7a4c14ada98..00000000000
--- a/src/interfaces/libpq/t/003_load_balance_host_list.pl
+++ /dev/null
@@ -1,94 +0,0 @@
-# Copyright (c) 2023-2025, PostgreSQL Global Development Group
-use strict;
-use warnings FATAL => 'all';
-use Config;
-use PostgreSQL::Test::Utils;
-use PostgreSQL::Test::Cluster;
-use Test::More;
-
-# This tests load balancing across the list of different hosts in the host
-# parameter of the connection string.
-
-# Cluster setup which is shared for testing both load balancing methods
-my $node1 = PostgreSQL::Test::Cluster->new('node1');
-my $node2 = PostgreSQL::Test::Cluster->new('node2', own_host => 1);
-my $node3 = PostgreSQL::Test::Cluster->new('node3', own_host => 1);
-
-# Create a data directory with initdb
-$node1->init();
-$node2->init();
-$node3->init();
-
-# Start the PostgreSQL server
-$node1->start();
-$node2->start();
-$node3->start();
-
-# Start the tests for load balancing method 1
-my $hostlist = $node1->host . ',' . $node2->host . ',' . $node3->host;
-my $portlist = $node1->port . ',' . $node2->port . ',' . $node3->port;
-
-$node1->connect_fails(
- "host=$hostlist port=$portlist load_balance_hosts=doesnotexist",
- "load_balance_hosts doesn't accept unknown values",
- expected_stderr => qr/invalid load_balance_hosts value: "doesnotexist"/);
-
-# load_balance_hosts=disable should always choose the first one.
-$node1->connect_ok(
- "host=$hostlist port=$portlist load_balance_hosts=disable",
- "load_balance_hosts=disable connects to the first node",
- sql => "SELECT 'connect1'",
- log_like => [qr/statement: SELECT 'connect1'/]);
-
-# Statistically the following loop with load_balance_hosts=random will almost
-# certainly connect at least once to each of the nodes. The chance of that not
-# happening is so small that it's negligible: (2/3)^50 = 1.56832855e-9
-foreach my $i (1 .. 50)
-{
- $node1->connect_ok(
- "host=$hostlist port=$portlist load_balance_hosts=random",
- "repeated connections with random load balancing",
- sql => "SELECT 'connect2'");
-}
-
-my $node1_occurrences = () =
- $node1->log_content() =~ /statement: SELECT 'connect2'/g;
-my $node2_occurrences = () =
- $node2->log_content() =~ /statement: SELECT 'connect2'/g;
-my $node3_occurrences = () =
- $node3->log_content() =~ /statement: SELECT 'connect2'/g;
-
-my $total_occurrences =
- $node1_occurrences + $node2_occurrences + $node3_occurrences;
-
-cmp_ok($node1_occurrences, '>', 1,
- "received at least one connection on node1");
-cmp_ok($node2_occurrences, '>', 1,
- "received at least one connection on node2");
-cmp_ok($node3_occurrences, '>', 1,
- "received at least one connection on node3");
-is($total_occurrences, 50, "received 50 connections across all nodes");
-
-$node1->stop();
-$node2->stop();
-
-# load_balance_hosts=disable should continue trying hosts until it finds a
-# working one.
-$node3->connect_ok(
- "host=$hostlist port=$portlist load_balance_hosts=disable",
- "load_balance_hosts=disable continues until it connects to the a working node",
- sql => "SELECT 'connect3'",
- log_like => [qr/statement: SELECT 'connect3'/]);
-
-# Also with load_balance_hosts=random we continue to the next nodes if previous
-# ones are down. Connect a few times to make sure it's not just lucky.
-foreach my $i (1 .. 5)
-{
- $node3->connect_ok(
- "host=$hostlist port=$portlist load_balance_hosts=random",
- "load_balance_hosts=random continues until it connects to the a working node",
- sql => "SELECT 'connect4'",
- log_like => [qr/statement: SELECT 'connect4'/]);
-}
-
-done_testing();
diff --git a/src/interfaces/libpq/t/004_load_balance_dns.pl b/src/interfaces/libpq/t/004_load_balance_dns.pl
deleted file mode 100644
index 2b4bd261c3d..00000000000
--- a/src/interfaces/libpq/t/004_load_balance_dns.pl
+++ /dev/null
@@ -1,144 +0,0 @@
-# Copyright (c) 2023-2025, PostgreSQL Global Development Group
-use strict;
-use warnings FATAL => 'all';
-use Config;
-use PostgreSQL::Test::Utils;
-use PostgreSQL::Test::Cluster;
-use Test::More;
-
-if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\bload_balance\b/)
-{
- plan skip_all =>
- 'Potentially unsafe test load_balance not enabled in PG_TEST_EXTRA';
-}
-
-# This tests loadbalancing based on a DNS entry that contains multiple records
-# for different IPs. Since setting up a DNS server is more effort than we
-# consider reasonable to run this test, this situation is instead imitated by
-# using a hosts file where a single hostname maps to multiple different IP
-# addresses. This test requires the administrator to add the following lines to
-# the hosts file (if we detect that this hasn't happened we skip the test):
-#
-# 127.0.0.1 pg-loadbalancetest
-# 127.0.0.2 pg-loadbalancetest
-# 127.0.0.3 pg-loadbalancetest
-#
-# Windows or Linux are required to run this test because these OSes allow
-# binding to 127.0.0.2 and 127.0.0.3 addresses by default, but other OSes
-# don't. We need to bind to different IP addresses, so that we can use these
-# different IP addresses in the hosts file.
-#
-# The hosts file needs to be prepared before running this test. We don't do it
-# on the fly, because it requires root permissions to change the hosts file. In
-# CI we set up the previously mentioned rules in the hosts file, so that this
-# load balancing method is tested.
-
-# Cluster setup which is shared for testing both load balancing methods
-my $can_bind_to_127_0_0_2 =
- $Config{osname} eq 'linux' || $PostgreSQL::Test::Utils::windows_os;
-
-# Checks for the requirements for testing load balancing method 2
-if (!$can_bind_to_127_0_0_2)
-{
- plan skip_all => 'load_balance test only supported on Linux and Windows';
-}
-
-my $hosts_path;
-if ($windows_os)
-{
- $hosts_path = 'c:\Windows\System32\Drivers\etc\hosts';
-}
-else
-{
- $hosts_path = '/etc/hosts';
-}
-
-my $hosts_content = PostgreSQL::Test::Utils::slurp_file($hosts_path);
-
-my $hosts_count = () =
- $hosts_content =~ /127\.0\.0\.[1-3] pg-loadbalancetest/g;
-if ($hosts_count != 3)
-{
- # Host file is not prepared for this test
- plan skip_all => "hosts file was not prepared for DNS load balance test";
-}
-
-$PostgreSQL::Test::Cluster::use_tcp = 1;
-$PostgreSQL::Test::Cluster::test_pghost = '127.0.0.1';
-my $port = PostgreSQL::Test::Cluster::get_free_port();
-my $node1 = PostgreSQL::Test::Cluster->new('node1', port => $port);
-my $node2 =
- PostgreSQL::Test::Cluster->new('node2', port => $port, own_host => 1);
-my $node3 =
- PostgreSQL::Test::Cluster->new('node3', port => $port, own_host => 1);
-
-# Create a data directory with initdb
-$node1->init();
-$node2->init();
-$node3->init();
-
-# Start the PostgreSQL server
-$node1->start();
-$node2->start();
-$node3->start();
-
-# load_balance_hosts=disable should always choose the first one.
-$node1->connect_ok(
- "host=pg-loadbalancetest port=$port load_balance_hosts=disable",
- "load_balance_hosts=disable connects to the first node",
- sql => "SELECT 'connect1'",
- log_like => [qr/statement: SELECT 'connect1'/]);
-
-
-# Statistically the following loop with load_balance_hosts=random will almost
-# certainly connect at least once to each of the nodes. The chance of that not
-# happening is so small that it's negligible: (2/3)^50 = 1.56832855e-9
-foreach my $i (1 .. 50)
-{
- $node1->connect_ok(
- "host=pg-loadbalancetest port=$port load_balance_hosts=random",
- "repeated connections with random load balancing",
- sql => "SELECT 'connect2'");
-}
-
-my $node1_occurrences = () =
- $node1->log_content() =~ /statement: SELECT 'connect2'/g;
-my $node2_occurrences = () =
- $node2->log_content() =~ /statement: SELECT 'connect2'/g;
-my $node3_occurrences = () =
- $node3->log_content() =~ /statement: SELECT 'connect2'/g;
-
-my $total_occurrences =
- $node1_occurrences + $node2_occurrences + $node3_occurrences;
-
-cmp_ok($node1_occurrences, '>', 1,
- "received at least one connection on node1");
-cmp_ok($node2_occurrences, '>', 1,
- "received at least one connection on node2");
-cmp_ok($node3_occurrences, '>', 1,
- "received at least one connection on node3");
-is($total_occurrences, 50, "received 50 connections across all nodes");
-
-$node1->stop();
-$node2->stop();
-
-# load_balance_hosts=disable should continue trying hosts until it finds a
-# working one.
-$node3->connect_ok(
- "host=pg-loadbalancetest port=$port load_balance_hosts=disable",
- "load_balance_hosts=disable continues until it connects to the a working node",
- sql => "SELECT 'connect3'",
- log_like => [qr/statement: SELECT 'connect3'/]);
-
-# Also with load_balance_hosts=random we continue to the next nodes if previous
-# ones are down. Connect a few times to make sure it's not just lucky.
-foreach my $i (1 .. 5)
-{
- $node3->connect_ok(
- "host=pg-loadbalancetest port=$port load_balance_hosts=random",
- "load_balance_hosts=random continues until it connects to the a working node",
- sql => "SELECT 'connect4'",
- log_like => [qr/statement: SELECT 'connect4'/]);
-}
-
-done_testing();
--
2.52.0
[text/x-patch] v5-0006-WIP-pytest-Add-some-SSL-client-tests.patch (18.7K, 7-v5-0006-WIP-pytest-Add-some-SSL-client-tests.patch)
download | inline diff:
From 99a8684e81edf2dd06b0f8b4b064e03b070ca6e8 Mon Sep 17 00:00:00 2001
From: Jacob Champion <[email protected]>
Date: Tue, 16 Dec 2025 09:30:55 +0100
Subject: [PATCH v5 6/7] WIP: pytest: Add some SSL client tests
This is a sample client-only test suite. It tests some handshake
failures against a mock server, as well as a full SSL handshake + empty
query + response.
pyca/cryptography is added as a new package dependency. Certificates for
testing are generated on the fly.
The mock design is threaded: the server socket is listening on a
background thread, and the test provides the server logic via a
callback. There is some additional work still needed to make this
production-ready; see the notes for _TCPServer.background(). (Currently,
an exception in the wrong place could result in a hang-until-timeout
rather than an immediate failure.)
TODOs:
- local_server and tcp_server_class are nearly identical and should
share code.
- fix exception-related timeouts for .background()
- figure out the proper use of "session" vs "module" scope
- ensure that pq.libpq unwinds (to close connections) before tcp_server;
see comment in test_server_with_ssl_disabled()
---
.cirrus.tasks.yml | 18 ++-
pyproject.toml | 8 +
src/test/ssl/Makefile | 2 +
src/test/ssl/meson.build | 6 +
src/test/ssl/pyt/conftest.py | 128 +++++++++++++++
src/test/ssl/pyt/test_client.py | 278 ++++++++++++++++++++++++++++++++
6 files changed, 434 insertions(+), 6 deletions(-)
create mode 100644 src/test/ssl/pyt/conftest.py
create mode 100644 src/test/ssl/pyt/test_client.py
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index a2c3febc30c..41d2a3c1867 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -229,6 +229,7 @@ task:
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
pkg install -y \
+ py311-cryptography \
py311-packaging \
py311-pytest
@@ -323,6 +324,7 @@ task:
setup_additional_packages_script: |
pkgin -y install \
+ py312-cryptography \
py312-packaging \
py312-test
ln -s /usr/pkg/bin/pytest-3.12 /usr/pkg/bin/pytest
@@ -346,8 +348,9 @@ task:
setup_additional_packages_script: |
pkg_add -I \
- py3-test \
- py3-packaging
+ py3-cryptography \
+ py3-packaging \
+ py3-test
# Always core dump to ${CORE_DUMP_DIR}
set_core_dump_script: sysctl -w kern.nosuidcoredump=2
<<: *openbsd_task_template
@@ -508,8 +511,9 @@ task:
setup_additional_packages_script: |
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get -y install \
- python3-pytest \
- python3-packaging
+ python3-cryptography \
+ python3-packaging \
+ python3-pytest
matrix:
# SPECIAL:
@@ -658,6 +662,7 @@ task:
CIRRUS_WORKING_DIR: ${HOME}/pgsql/
CCACHE_DIR: ${HOME}/ccache
MACPORTS_CACHE: ${HOME}/macports-cache
+ PYTEST_DEBUG_TEMPROOT: /tmp # default is too long for UNIX sockets on Mac
MESON_FEATURES: >-
-Dbonjour=enabled
@@ -678,6 +683,7 @@ task:
p5.34-io-tty
p5.34-ipc-run
python312
+ py312-cryptography
py312-packaging
py312-pytest
tcl
@@ -816,7 +822,7 @@ task:
# XXX Does Chocolatey really not have any Python package installers?
setup_additional_packages_script: |
REM choco install -y --no-progress ...
- pip3 install --user packaging pytest
+ pip3 install --user cryptography packaging pytest
setup_hosts_file_script: |
echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
@@ -879,7 +885,7 @@ task:
folder: ${CCACHE_DIR}
setup_additional_packages_script: |
- C:\msys64\usr\bin\pacman.exe -S --noconfirm mingw-w64-ucrt-x86_64-python-pytest
+ C:\msys64\usr\bin\pacman.exe -S --noconfirm mingw-w64-ucrt-x86_64-python-cryptography mingw-w64-ucrt-x86_64-python-pytest
mingw_info_script: |
%BASH% -c "where gcc"
diff --git a/pyproject.toml b/pyproject.toml
index 4628d2274e0..00c8ae88583 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -12,6 +12,14 @@ dependencies = [
# Any other dependencies are effectively optional (added below). We import
# these libraries using pytest.importorskip(). So tests will be skipped if
# they are not available.
+
+ # Notes on the cryptography package:
+ # - 3.3.2 is shipped on Debian bullseye.
+ # - 3.4.x drops support for Python 2, making it a version of note for older LTS
+ # distros.
+ # - 35.x switched versioning schemes and moved to Rust parsing.
+ # - 40.x is the last version supporting Python 3.6.
+ "cryptography >= 3.3.2",
]
[tool.pytest.ini_options]
diff --git a/src/test/ssl/Makefile b/src/test/ssl/Makefile
index e8a1639db2d..895ea5ea41c 100644
--- a/src/test/ssl/Makefile
+++ b/src/test/ssl/Makefile
@@ -30,6 +30,8 @@ clean distclean:
# Doesn't depend on sslfiles because we don't rebuild them by default
check:
$(prove_check)
+ # XXX these suites should run independently, not serially
+ $(pytest_check)
installcheck:
$(prove_installcheck)
diff --git a/src/test/ssl/meson.build b/src/test/ssl/meson.build
index d8e0fb518e0..a0ee2af0899 100644
--- a/src/test/ssl/meson.build
+++ b/src/test/ssl/meson.build
@@ -15,4 +15,10 @@ tests += {
't/003_sslinfo.pl',
],
},
+ 'pytest': {
+ 'tests': [
+ 'pyt/test_client.py',
+ 'pyt/test_server.py',
+ ],
+ },
}
diff --git a/src/test/ssl/pyt/conftest.py b/src/test/ssl/pyt/conftest.py
new file mode 100644
index 00000000000..870f738ac44
--- /dev/null
+++ b/src/test/ssl/pyt/conftest.py
@@ -0,0 +1,128 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import datetime
+import re
+import subprocess
+import tempfile
+from collections import namedtuple
+
+import pytest
+
+
[email protected](scope="session")
+def cryptography():
+ return pytest.importorskip("cryptography", "3.3.2")
+
+
+Cert = namedtuple("Cert", "cert, certpath, key, keypath")
+
+
[email protected](scope="session")
+def certs(cryptography, tmp_path_factory):
+ """
+ Caches commonly used certificates at the session level, and provides a way
+ to create new ones.
+
+ - certs.ca: the root CA certificate
+
+ - certs.server: the "standard" server certficate, signed by certs.ca
+
+ - certs.server_host: the hostname of the certs.server certificate
+
+ - certs.new(): creates a custom certificate, signed by certs.ca
+ """
+
+ from cryptography import x509
+ from cryptography.hazmat.primitives import hashes, serialization
+ from cryptography.hazmat.primitives.asymmetric import rsa
+ from cryptography.x509.oid import NameOID
+
+ tmpdir = tmp_path_factory.mktemp("test-certs")
+
+ class _Certs:
+ def __init__(self):
+ self.ca = self.new(
+ x509.Name(
+ [x509.NameAttribute(NameOID.COMMON_NAME, "PG pytest CA")],
+ ),
+ ca=True,
+ )
+
+ self.server_host = "example.org"
+ self.server = self.new(
+ x509.Name(
+ [x509.NameAttribute(NameOID.COMMON_NAME, self.server_host)],
+ )
+ )
+
+ def new(self, subject: x509.Name, *, ca=False) -> Cert:
+ """
+ Creates and signs a new Cert with the given subject name. If ca is
+ True, the certificate will be self-signed; otherwise the certificate
+ is signed by self.ca.
+ """
+ key = rsa.generate_private_key(
+ public_exponent=65537,
+ key_size=2048,
+ )
+
+ builder = x509.CertificateBuilder()
+ now = datetime.datetime.now(datetime.timezone.utc)
+
+ builder = (
+ builder.subject_name(subject)
+ .public_key(key.public_key())
+ .serial_number(x509.random_serial_number())
+ .not_valid_before(now)
+ .not_valid_after(now + datetime.timedelta(hours=1))
+ )
+
+ if ca:
+ builder = builder.issuer_name(subject)
+ else:
+ builder = builder.issuer_name(self.ca.cert.subject)
+
+ builder = builder.add_extension(
+ x509.BasicConstraints(ca=ca, path_length=None),
+ critical=True,
+ )
+
+ cert = builder.sign(
+ private_key=key if ca else self.ca.key,
+ algorithm=hashes.SHA256(),
+ )
+
+ # Dump the certificate and key to file.
+ keypath = self._tofile(
+ key.private_bytes(
+ serialization.Encoding.PEM,
+ serialization.PrivateFormat.PKCS8,
+ serialization.NoEncryption(),
+ ),
+ suffix=".key",
+ )
+ certpath = self._tofile(
+ cert.public_bytes(serialization.Encoding.PEM),
+ suffix="-ca.crt" if ca else ".crt",
+ )
+
+ return Cert(
+ cert=cert,
+ certpath=certpath,
+ key=key,
+ keypath=keypath,
+ )
+
+ def _tofile(self, data: bytes, *, suffix) -> str:
+ """
+ Dumps data to a file on disk with the requested suffix and returns
+ the path. The file is located somewhere in pytest's temporary
+ directory root.
+ """
+ f = tempfile.NamedTemporaryFile(suffix=suffix, dir=tmpdir, delete=False)
+ with f:
+ f.write(data)
+
+ return f.name
+
+ return _Certs()
diff --git a/src/test/ssl/pyt/test_client.py b/src/test/ssl/pyt/test_client.py
new file mode 100644
index 00000000000..556bad33bf8
--- /dev/null
+++ b/src/test/ssl/pyt/test_client.py
@@ -0,0 +1,278 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import contextlib
+import ctypes
+import socket
+import ssl
+import struct
+import threading
+from typing import Callable
+
+import pytest
+
+import pypg
+from libpq import LibpqError, ExecStatus
+
+# This suite opens up local TCP ports and is hidden behind PG_TEST_EXTRA=ssl.
+pytestmark = pypg.require_test_extras("ssl")
+
+
[email protected](scope="session", autouse=True)
+def skip_if_no_ssl_support(libpq_handle):
+ """Skips tests if SSL support is not configured."""
+
+ # Declare PQsslAttribute().
+ PQsslAttribute = libpq_handle.PQsslAttribute
+ PQsslAttribute.restype = ctypes.c_char_p
+ PQsslAttribute.argtypes = [ctypes.c_void_p, ctypes.c_char_p]
+
+ if not PQsslAttribute(None, b"library"):
+ pytest.skip("requires SSL support to be configured")
+
+
+#
+# Test Fixtures
+#
+
+
[email protected]
+def tcp_server_class(remaining_timeout):
+ """
+ Metafixture to combine related logic for tcp_server and ssl_server.
+
+ TODO: combine with test_libpq.local_server
+ """
+
+ class _TCPServer(contextlib.ExitStack):
+ """
+ Implementation class for tcp_server. See .background() for the primary
+ entry point for tests. Postgres clients may connect to this server via
+ **tcp_server.conninfo.
+
+ _TCPServer derives from contextlib.ExitStack to provide easy cleanup of
+ associated resources; see the documentation for that class for a full
+ explanation.
+ """
+
+ def __init__(self):
+ super().__init__()
+
+ self._thread = None
+ self._thread_exc = None
+ self._listener = self.enter_context(
+ socket.socket(socket.AF_INET, socket.SOCK_STREAM),
+ )
+
+ self._bind_and_listen()
+ sockname = self._listener.getsockname()
+ self.conninfo = dict(
+ hostaddr=sockname[0],
+ port=sockname[1],
+ )
+
+ def _bind_and_listen(self):
+ """
+ Does the actual work of binding the socket and listening for
+ connections.
+
+ The listen backlog is currently hardcoded to one.
+ """
+ self._listener.bind(("127.0.0.1", 0))
+ self._listener.listen(1)
+
+ def background(self, fn: Callable[[socket.socket], None]) -> None:
+ """
+ Accepts a client connection on a background thread and passes it to
+ the provided callback. Any exceptions raised from the callback will
+ be re-raised on the main thread during fixture teardown.
+
+ Blocking operations on the connected socket default to using the
+ remaining_timeout(), though this can be changed by the test via the
+ socket's .settimeout().
+ """
+
+ def _bg():
+ try:
+ self._listener.settimeout(remaining_timeout())
+ sock, _ = self._listener.accept()
+
+ with sock:
+ sock.settimeout(remaining_timeout())
+ fn(sock)
+
+ except Exception as e:
+ # Save the exception for re-raising on the main thread.
+ self._thread_exc = e
+
+ # TODO: rather than using callback(), consider explicitly signaling
+ # the fn() implementation to stop early if we get an exception.
+ # Otherwise we'll hang until the end of the timeout.
+ self._thread = threading.Thread(target=_bg)
+ self.callback(self._join)
+
+ self._thread.start()
+
+ def _join(self):
+ """
+ Waits for the background thread to finish and raises any thrown
+ exception. This is called during fixture teardown.
+ """
+ # Give a little bit of wiggle room on the join timeout, since we're
+ # racing against the test's own use of remaining_timeout(). (It's
+ # preferable to let tests report timeouts; the stack traces will
+ # help with debugging.)
+ self._thread.join(remaining_timeout() + 1)
+ if self._thread.is_alive():
+ raise TimeoutError("background thread is still running after timeout")
+
+ if self._thread_exc is not None:
+ raise self._thread_exc
+
+ return _TCPServer
+
+
[email protected]
+def tcp_server(tcp_server_class):
+ """
+ Opens up a local TCP socket for mocking a Postgres server on a background
+ thread. See the _TCPServer API for usage.
+ """
+ with tcp_server_class() as s:
+ yield s
+
+
[email protected]
+def ssl_server(tcp_server_class, certs):
+ """
+ Like tcp_server, but with an additional .background_ssl() method which will
+ perform a SSLRequest handshake on the socket before handing the connection
+ to the test callback.
+
+ This server uses certs.server as its identity.
+ """
+
+ class _SSLServer(tcp_server_class):
+ def __init__(self):
+ super().__init__()
+
+ self.conninfo["host"] = certs.server_host
+
+ self._ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+ self._ctx.load_cert_chain(certs.server.certpath, certs.server.keypath)
+
+ def background_ssl(self, fn: Callable[[ssl.SSLSocket], None]) -> None:
+ """
+ Invokes a server callback as with .background(), but an SSLRequest
+ handshake is performed first, and the socket provided to the
+ callback has been wrapped in an OpenSSL layer.
+ """
+
+ def handshake(s: socket.socket):
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+
+ # Make sure we get an SSLRequest.
+ version = struct.unpack("!HH", s.recv(4))
+ assert version == (1234, 5679)
+ assert pktlen == 8
+
+ # Accept the SSLRequest.
+ s.send(b"S")
+
+ with self._ctx.wrap_socket(s, server_side=True) as wrapped:
+ fn(wrapped)
+
+ self.background(handshake)
+
+ with _SSLServer() as s:
+ yield s
+
+
+#
+# Tests
+#
+
+
[email protected]("sslmode", ("require", "verify-ca", "verify-full"))
+def test_server_with_ssl_disabled(connect, tcp_server, certs, sslmode):
+ """
+ Make sure client refuses to talk to non-SSL servers with stricter
+ sslmodes.
+ """
+
+ def refuse_ssl(s: socket.socket):
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+
+ # Make sure we get an SSLRequest.
+ version = struct.unpack("!HH", s.recv(4))
+ assert version == (1234, 5679)
+ assert pktlen == 8
+
+ # Refuse the SSLRequest.
+ s.send(b"N")
+
+ # Wait for the client to close the connection.
+ assert not s.recv(1), "client sent unexpected data"
+
+ tcp_server.background(refuse_ssl)
+
+ with pytest.raises(LibpqError, match="server does not support SSL"):
+ connect(
+ **tcp_server.conninfo,
+ sslrootcert=certs.ca.certpath,
+ sslmode=sslmode,
+ )
+
+
+def test_verify_full_connection(connect, ssl_server, certs):
+ """Completes a verify-full connection and empty query."""
+
+ def handle_empty_query(s: ssl.SSLSocket):
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+
+ # Check the startup packet version, then discard the remainder.
+ version = struct.unpack("!HH", s.recv(4))
+ assert version == (3, 0)
+ s.recv(pktlen - 8)
+
+ # Send the required litany of server messages.
+ s.send(struct.pack("!cII", b"R", 8, 0)) # AuthenticationOK
+
+ # ParameterStatus: client_encoding
+ key = b"client_encoding\0"
+ val = b"UTF-8\0"
+ s.send(struct.pack("!cI", b"S", 4 + len(key) + len(val)) + key + val)
+
+ # ParameterStatus: DateStyle
+ key = b"DateStyle\0"
+ val = b"ISO, MDY\0"
+ s.send(struct.pack("!cI", b"S", 4 + len(key) + len(val)) + key + val)
+
+ s.send(struct.pack("!cIII", b"K", 12, 1234, 1234)) # BackendKeyData
+ s.send(struct.pack("!cIc", b"Z", 5, b"I")) # ReadyForQuery
+
+ # Expect an empty query.
+ pkttype = s.recv(1)
+ assert pkttype == b"Q"
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+ assert s.recv(pktlen - 4) == b"\0"
+
+ # Send an EmptyQueryResponse+ReadyForQuery.
+ s.send(struct.pack("!cI", b"I", 4))
+ s.send(struct.pack("!cIc", b"Z", 5, b"I"))
+
+ # libpq should terminate and close the connection.
+ assert s.recv(1) == b"X"
+ pktlen = struct.unpack("!I", s.recv(4))[0]
+ assert pktlen == 4
+
+ assert not s.recv(1), "client sent unexpected data"
+
+ ssl_server.background_ssl(handle_empty_query)
+
+ conn = connect(
+ **ssl_server.conninfo,
+ sslrootcert=certs.ca.certpath,
+ sslmode="verify-full",
+ )
+ with conn:
+ assert conn.exec("").status() == ExecStatus.PGRES_EMPTY_QUERY
--
2.52.0
[text/x-patch] v5-0007-WIP-pytest-Add-some-server-side-SSL-tests.patch (8.5K, 8-v5-0007-WIP-pytest-Add-some-server-side-SSL-tests.patch)
download | inline diff:
From 2d78377ddcb558debee06040dbd58e2012dd9c8a Mon Sep 17 00:00:00 2001
From: Jacob Champion <[email protected]>
Date: Tue, 16 Dec 2025 09:31:46 +0100
Subject: [PATCH v5 7/7] WIP: pytest: Add some server-side SSL tests
In the same vein as the previous commit, this is a server-only test
suite operating against a mock client. The test itself is a heavily
parameterized check for direct-SSL handshake behavior, using a
combination of "standard" and "custom" certificates via the certs
fixture.
installcheck is currently unsupported, but the architecture has some
extension points that should make it possible later. For now, a new
server is always started for the test session.
TODOs:
- improve remaining_timeout() integration with socket operations; at the
moment, the timeout resets on every call rather than decrementing
---
src/test/ssl/pyt/conftest.py | 50 ++++++++++
src/test/ssl/pyt/test_server.py | 161 ++++++++++++++++++++++++++++++++
2 files changed, 211 insertions(+)
create mode 100644 src/test/ssl/pyt/test_server.py
diff --git a/src/test/ssl/pyt/conftest.py b/src/test/ssl/pyt/conftest.py
index 870f738ac44..d121724800b 100644
--- a/src/test/ssl/pyt/conftest.py
+++ b/src/test/ssl/pyt/conftest.py
@@ -126,3 +126,53 @@ def certs(cryptography, tmp_path_factory):
return f.name
return _Certs()
+
+
[email protected](scope="module", autouse=True)
+def ssl_setup(pg_server_module, certs, datadir):
+ """
+ Sets up required server settings for all tests in this module.
+ """
+ try:
+ with pg_server_module.restarting() as s:
+ s.conf.set(
+ ssl="on",
+ ssl_ca_file=certs.ca.certpath,
+ ssl_cert_file=certs.server.certpath,
+ ssl_key_file=certs.server.keypath,
+ )
+
+ # Reject by default.
+ s.hba.prepend("hostssl all all all reject")
+
+ except subprocess.CalledProcessError:
+ # This is a decent place to skip if the server isn't set up for SSL.
+ logpath = datadir / "postgresql.log"
+ unsupported = re.compile("SSL is not supported")
+
+ with open(logpath, "r") as log:
+ for line in log:
+ if unsupported.search(line):
+ pytest.skip("the server does not support SSL")
+
+ # Some other error happened.
+ raise
+
+ users = pg_server_module.create_users("ssl")
+ dbs = pg_server_module.create_dbs("ssl")
+
+ return (users, dbs)
+
+
[email protected](scope="module")
+def client_cert(ssl_setup, certs):
+ """
+ Creates a Cert for the "ssl" user.
+ """
+ from cryptography import x509
+ from cryptography.x509.oid import NameOID
+
+ users, _ = ssl_setup
+ user = users["ssl"]
+
+ return certs.new(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, user)]))
diff --git a/src/test/ssl/pyt/test_server.py b/src/test/ssl/pyt/test_server.py
new file mode 100644
index 00000000000..d5cb14b6c9a
--- /dev/null
+++ b/src/test/ssl/pyt/test_server.py
@@ -0,0 +1,161 @@
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+import re
+import socket
+import ssl
+import struct
+
+import pytest
+
+import pypg
+
+# This suite opens up local TCP ports and is hidden behind PG_TEST_EXTRA=ssl.
+pytestmark = pypg.require_test_extras("ssl")
+
+# For use with the `creds` parameter below.
+CLIENT = "client"
+SERVER = "server"
+
+
+# fmt: off
[email protected](
+ "auth_method, creds, expected_error",
+[
+ # Trust allows anything.
+ ("trust", None, None),
+ ("trust", CLIENT, None),
+ ("trust", SERVER, None),
+
+ # verify-ca allows any CA-signed certificate.
+ ("trust clientcert=verify-ca", None, "requires a valid client certificate"),
+ ("trust clientcert=verify-ca", CLIENT, None),
+ ("trust clientcert=verify-ca", SERVER, None),
+
+ # cert and verify-full allow only the correct certificate.
+ ("trust clientcert=verify-full", None, "requires a valid client certificate"),
+ ("trust clientcert=verify-full", CLIENT, None),
+ ("trust clientcert=verify-full", SERVER, "authentication failed for user"),
+ ("cert", None, "requires a valid client certificate"),
+ ("cert", CLIENT, None),
+ ("cert", SERVER, "authentication failed for user"),
+],
+)
+# fmt: on
+def test_direct_ssl_certificate_authentication(
+ pg,
+ ssl_setup,
+ certs,
+ client_cert,
+ remaining_timeout,
+ # test parameters
+ auth_method,
+ creds,
+ expected_error,
+):
+ """
+ Tests direct SSL connections with various client-certificate/HBA
+ combinations.
+ """
+
+ # Set up the HBA as desired by the test.
+ users, dbs = ssl_setup
+
+ user = users["ssl"]
+ db = dbs["ssl"]
+
+ with pg.reloading() as s:
+ s.hba.prepend(
+ ["hostssl", db, user, "127.0.0.1/32", auth_method],
+ ["hostssl", db, user, "::1/128", auth_method],
+ )
+
+ # Configure the SSL settings for the client.
+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
+ ctx.load_verify_locations(cafile=certs.ca.certpath)
+ ctx.set_alpn_protocols(["postgresql"]) # for direct SSL
+
+ # Load up a client certificate if required by the test.
+ if creds == CLIENT:
+ ctx.load_cert_chain(client_cert.certpath, client_cert.keypath)
+ elif creds == SERVER:
+ # Using a server certificate as the client credential is expected to
+ # work only for clientcert=verify-ca (and `trust`, naturally).
+ ctx.load_cert_chain(certs.server.certpath, certs.server.keypath)
+
+ # Make a direct SSL connection. There's no SSLRequest in the handshake; we
+ # simply wrap a TCP connection with OpenSSL.
+ addr = (pg.hostaddr, pg.port)
+ with socket.create_connection(addr) as s:
+ s.settimeout(remaining_timeout()) # XXX this resets every operation
+
+ with ctx.wrap_socket(s, server_hostname=certs.server_host) as conn:
+ # Build and send the startup packet.
+ startup_options = dict(
+ user=user,
+ database=db,
+ application_name="pytest",
+ )
+
+ payload = b""
+ for k, v in startup_options.items():
+ payload += k.encode() + b"\0"
+ payload += str(v).encode() + b"\0"
+ payload += b"\0" # null terminator
+
+ pktlen = 4 + 4 + len(payload)
+ conn.send(struct.pack("!IHH", pktlen, 3, 0) + payload)
+
+ if not expected_error:
+ # Expect an AuthenticationOK to come back.
+ pkttype, pktlen = struct.unpack("!cI", conn.recv(5))
+ assert pkttype == b"R"
+ assert pktlen == 8
+
+ authn_result = struct.unpack("!I", conn.recv(4))[0]
+ assert authn_result == 0
+
+ # Read and discard to ReadyForQuery.
+ while True:
+ pkttype, pktlen = struct.unpack("!cI", conn.recv(5))
+ payload = conn.recv(pktlen - 4)
+
+ if pkttype == b"Z":
+ assert payload == b"I"
+ break
+
+ # Send an empty query.
+ conn.send(struct.pack("!cI", b"Q", 5) + b"\0")
+
+ # Expect EmptyQueryResponse+ReadyForQuery.
+ pkttype, pktlen = struct.unpack("!cI", conn.recv(5))
+ assert pkttype == b"I"
+ assert pktlen == 4
+
+ pkttype, pktlen = struct.unpack("!cI", conn.recv(5))
+ assert pkttype == b"Z"
+
+ payload = conn.recv(pktlen - 4)
+ assert payload == b"I"
+
+ else:
+ # Match the expected authentication error.
+ pkttype, pktlen = struct.unpack("!cI", conn.recv(5))
+ assert pkttype == b"E"
+
+ payload = conn.recv(pktlen - 4)
+ msg = None
+
+ for component in payload.split(b"\0"):
+ if not component:
+ break # end of message
+
+ key, val = component[:1], component[1:]
+ if key == b"S":
+ assert val == b"FATAL"
+ elif key == b"M":
+ msg = val.decode()
+
+ assert re.search(expected_error, msg), "server error did not match"
+
+ # Terminate.
+ conn.send(struct.pack("!cI", b"X", 4))
--
2.52.0
view thread (79+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: RFC: adding pytest as a supported test framework
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox