public inbox for [email protected]
help / color / mirror / Atom feedBUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
10+ messages / 3 participants
[nested] [flat]
* BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-26 19:00 PG Bug reporting form <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: PG Bug reporting form @ 2026-05-26 19:00 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
The following bug has been logged on the website:
Bug reference: 19494
Logged by: Alexander Lakhin
Email address: [email protected]
PostgreSQL version: 18.4
Operating system: Ubuntu 24.04
Description:
The following psql script:
CREATE TABLE t(a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
\startpipeline
INSERT INTO t VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
\endpipeline
results in:
a
---
1
1
(2 rows)
ERROR: duplicate key value violates unique constraint "t_pkey"
DETAIL: Key (a)=(1) already exists.
psql: common.c:1503: discardAbortedPipelineResults: Assertion
`pset.available_results > 0' failed.
Program terminated with signal SIGABRT, Aborted.
(gdb) bt
#0 __pthread_kill_implementation (threadid=281473811984096,
signo=signo@entry=6, no_tid=no_tid@entry=0)
at ./nptl/pthread_kill.c:44
#1 0x0000ffffba60b718 [PAC] in __pthread_kill_internal (threadid=<optimized
out>, signo=6) at ./nptl/pthread_kill.c:89
#2 0x0000ffffba5b757c in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3 0x0000ffffba5a1d48 [PAC] in __GI_abort () at ./stdlib/abort.c:77
#4 0x0000ffffba5b05e0 [PAC] in __assert_fail_base (fmt=<optimized out>,
assertion=<optimized out>,
file=0xaaaac5a317a8 "common.c", line=1503, function=<optimized out>) at
./assert/assert.c:118
#5 0x0000aaaac59d3ffc [PAC] in discardAbortedPipelineResults () at
common.c:1503
#6 0x0000aaaac59d4b28 in ExecQueryAndProcessResults (query=0xaaaae5929250
"INSERT INTO t VALUES ($1), ($1) RETURNING * ",
elapsed_msec=0xffffe4d657b0, svpt_gone_p=0xffffe4d657af, is_watch=false,
min_rows=0, opt=0x0, printQueryFout=0x0)
at common.c:1830
#7 0x0000aaaac59d37b0 in SendQuery (query=0xaaaae5929250 "INSERT INTO t
VALUES ($1), ($1) RETURNING * ") at common.c:1214
#8 0x0000aaaac59ee4f8 in MainLoop (source=0xffffba730870 <_IO_2_1_stdin_>)
at mainloop.c:515
#9 0x0000aaaac59cd488 in process_file (filename=0x0,
use_relative_path=false) at command.c:4977
#10 0x0000aaaac59fbbcc in main (argc=10, argv=0xffffe4d65f68) at
startup.c:424
Reproduced starting from 41625ab8e (with s/\sendpipeline/\g/ before
17caf6644).
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-28 03:51 Michael Paquier <[email protected]>
parent: PG Bug reporting form <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Michael Paquier @ 2026-05-28 03:51 UTC (permalink / raw)
To: [email protected]; [email protected]
On Tue, May 26, 2026 at 07:00:01PM +0000, PG Bug reporting form wrote:
> The following psql script:
> CREATE TABLE t(a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
> \startpipeline
> INSERT INTO t VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
> \endpipeline
>
> psql: common.c:1503: discardAbortedPipelineResults: Assertion
> `pset.available_results > 0' failed.
That's one assertion, and with a bit more imagination for a deferred
constraint, I can trigger a second one for the number of syncs when at
the end of a pipeline:
\startpipeline
INSERT INTO pipeline_defer_tab VALUES ($1), ($1) \bind 1 \sendpipeline
\syncpipeline
SELECT 1 \bind \sendpipeline
\endpipeline
#5 0x000000000041bb0a in ExecQueryAndProcessResults (query=0x5db560
"SELECT 1 ", elapsed_msec=0x7fffffffc660, svpt_gone_p=0x7fffffffc65f,
is_watch=false, min_rows=0, opt=0x0, printQueryFout=0x0) at
common.c:2190
2190 Assert(pset.piped_syncs == 0);
(gdb) p pset.piped_syncs
$1 = 1
I am completely sure yet, but it looks like we will need to be smarter
with the handling of the number of piped commands by tracking them
across the syncs in the shape of a queue, or something like that? So
it feels like we need to think harder about the tracking of this
activity depending on the state of the pipeline we're in. Or we could
lift some of these assertions, but that would not be right to me.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-28 05:26 Michael Paquier <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Michael Paquier @ 2026-05-28 05:26 UTC (permalink / raw)
To: [email protected]; [email protected]
On Thu, May 28, 2026 at 12:51:38PM +0900, Michael Paquier wrote:
> I am completely sure yet, but it looks like we will need to be smarter
> with the handling of the number of piped commands by tracking them
> across the syncs in the shape of a queue, or something like that? So
> it feels like we need to think harder about the tracking of this
> activity depending on the state of the pipeline we're in. Or we could
> lift some of these assertions, but that would not be right to me.
Hmm. Taking a step back this would be overcomplicating things. As
long as we are careful to consume the synced results still in a
pipeline, it looks like we should be fine. While digging into it, I
have found a third assertion that was triggerable with
available_results at the end of the pipeline, once I began mixing
\getresults with a deferred error.
This stuff is tricky enough that I may not have overseen all the
patterns possible, of course, at least this is progress.
Alexander, what do you think?
--
Michael
From f9ce93fc1e9bb47f4a50605a2b836bb9cc278989 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Thu, 28 May 2026 14:20:47 +0900
Subject: [PATCH] psql: Fix failures with deferred errors in pipelines
---
src/bin/psql/common.c | 48 ++++++++---
src/test/regress/expected/psql_pipeline.out | 91 +++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 39 +++++++++
3 files changed, 166 insertions(+), 12 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..3698f37c742c 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,19 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself.
+ * Such errors are not counted in available_results because they
+ * are not associated with a piped command. In that case, skip
+ * the counter decrements and continue to find the sync result.
+ */
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2175,14 +2183,30 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by Sync processing itself can leave some of
+ * them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ continue;
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
- Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..66e1e4f2ef5e 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,96 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..ec68e48e25aa 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,44 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
Attachments:
[text/plain] 0001-psql-Fix-failures-with-deferred-errors-in-pipelines.patch (7.3K, 2-0001-psql-Fix-failures-with-deferred-errors-in-pipelines.patch)
download | inline diff:
From f9ce93fc1e9bb47f4a50605a2b836bb9cc278989 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Thu, 28 May 2026 14:20:47 +0900
Subject: [PATCH] psql: Fix failures with deferred errors in pipelines
---
src/bin/psql/common.c | 48 ++++++++---
src/test/regress/expected/psql_pipeline.out | 91 +++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 39 +++++++++
3 files changed, 166 insertions(+), 12 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..3698f37c742c 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,19 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself.
+ * Such errors are not counted in available_results because they
+ * are not associated with a piped command. In that case, skip
+ * the counter decrements and continue to find the sync result.
+ */
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2175,14 +2183,30 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by Sync processing itself can leave some of
+ * them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ continue;
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
- Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..66e1e4f2ef5e 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,96 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..ec68e48e25aa 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,44 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
[application/pgp-signature] signature.asc (833B, 3-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-29 04:00 Alexander Lakhin <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Alexander Lakhin @ 2026-05-29 04:00 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; [email protected]
Hello Michael,
28.05.2026 08:26, Michael Paquier wrote:
> On Thu, May 28, 2026 at 12:51:38PM +0900, Michael Paquier wrote:
>> I am completely sure yet, but it looks like we will need to be smarter
>> with the handling of the number of piped commands by tracking them
>> across the syncs in the shape of a queue, or something like that? So
>> it feels like we need to think harder about the tracking of this
>> activity depending on the state of the pipeline we're in. Or we could
>> lift some of these assertions, but that would not be right to me.
> Hmm. Taking a step back this would be overcomplicating things. As
> long as we are careful to consume the synced results still in a
> pipeline, it looks like we should be fine. While digging into it, I
> have found a third assertion that was triggerable with
> available_results at the end of the pipeline, once I began mixing
> \getresults with a deferred error.
>
> This stuff is tricky enough that I may not have overseen all the
> patterns possible, of course, at least this is progress.
>
> Alexander, what do you think?
While testing the patch, I've observed apparently new anomaly. psql got
stuck inside this loop:
if (end_pipeline)
{
/*
* Reset available/requested results. Normally these are already 0,
* but an error generated by Sync processing itself can leave some of
* them behind. Consume them before exiting pipeline mode.
*/
while (pset.piped_syncs > 0)
{
PGresult *remaining = PQgetResult(pset.db);
if (remaining == NULL)
continue;
...
}
it's happening upon/after postgres process termination, so PQgetResult()
returns NULL, pset.piped_syncs == 1. I need more time to look deeper and
to come with a reproducer, but maybe you can already see what's wrong.
Best regards,
Alexander
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-29 05:00 Michael Paquier <[email protected]>
parent: Alexander Lakhin <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Michael Paquier @ 2026-05-29 05:00 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: [email protected]
On Fri, May 29, 2026 at 07:00:01AM +0300, Alexander Lakhin wrote:
> it's happening upon/after postgres process termination, so PQgetResult()
> returns NULL, pset.piped_syncs == 1. I need more time to look deeper and
> to come with a reproducer, but maybe you can already see what's wrong.
Yeah, I do. Nice catch. See this sequence to reproduce the problem:
\startpipeline
INSERT INTO psql_pipeline_defer VALUES (1), (1) \bind \sendpipeline
\syncpipeline
SELECT pg_terminate_backend(pg_backend_pid()) \bind \sendpipeline
SELECT 1 \bind \sendpipeline
\endpipeline
When ending the pipeline the loop consuming the results is stuck, so
we could check the connection state. We are going to enter in a
freeze of the branches due to beta1 next week, so let's take our time.
Please feel to use the v2 attached for your tests. I am also testing
it more on my side.
--
Michael
From c7e13a8826f25b3c99fe554f7e2c52a56bbea626 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Fri, 29 May 2026 13:58:29 +0900
Subject: [PATCH v2] psql: Fix issues with deferred errors in pipelines
When an error is raised while processing a Sync message in a pipeline, e
like a deferred constraint violation, the error was not associated with
the piped command and was not counted in available_results. This caused
assertion failures in discardAbortedPipelineResults(), keeping an
incorrect state at pipeline exit, because the code assumed that
the number of available and requested results would always be positive,
expecting all the counters to be 0 at the end of a pipeline.
This commit switches discardAbortedPipelineResults() and
ExecQueryAndProcessResults() to take a softer approach when consuming
and draining the results after an error since the error generated by a
Sync are not tracked in the result counters (well, we could perhaps do
that, but I am not convinced that this is worth the complication).
The reporter has shown one problematic assertion failure. While
investigating more this issue I have bumped into two more. All these
cases are covered by the regression tests added in this commit, plus
some bonuses.
Reported-by: Alexander Lakhin <[email protected]>
Author: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 18
---
src/bin/psql/common.c | 51 ++++++--
src/test/regress/expected/psql_pipeline.out | 124 ++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 63 ++++++++++
3 files changed, 227 insertions(+), 11 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..c021eb97552a 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,19 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself. Such
+ * errors are not counted in available_results because they are
+ * not associated with a piped command. In that case, skip the
+ * counter decrements and continue to find the Sync result.
+ */
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2175,14 +2183,35 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by a Sync processing itself can leave some
+ * of them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ {
+ if (!ConnectionUp())
+ break;
+ continue;
+ }
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..a931d63cafe7 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,129 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+ ?column?
+----------
+ partial
+(1 row)
+
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+\endpipeline
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..468ef1d090b6 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,68 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+\getresults
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
Attachments:
[text/plain] v2-0001-psql-Fix-issues-with-deferred-errors-in-pipelines.patch (10.5K, 2-v2-0001-psql-Fix-issues-with-deferred-errors-in-pipelines.patch)
download | inline diff:
From c7e13a8826f25b3c99fe554f7e2c52a56bbea626 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Fri, 29 May 2026 13:58:29 +0900
Subject: [PATCH v2] psql: Fix issues with deferred errors in pipelines
When an error is raised while processing a Sync message in a pipeline, e
like a deferred constraint violation, the error was not associated with
the piped command and was not counted in available_results. This caused
assertion failures in discardAbortedPipelineResults(), keeping an
incorrect state at pipeline exit, because the code assumed that
the number of available and requested results would always be positive,
expecting all the counters to be 0 at the end of a pipeline.
This commit switches discardAbortedPipelineResults() and
ExecQueryAndProcessResults() to take a softer approach when consuming
and draining the results after an error since the error generated by a
Sync are not tracked in the result counters (well, we could perhaps do
that, but I am not convinced that this is worth the complication).
The reporter has shown one problematic assertion failure. While
investigating more this issue I have bumped into two more. All these
cases are covered by the regression tests added in this commit, plus
some bonuses.
Reported-by: Alexander Lakhin <[email protected]>
Author: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 18
---
src/bin/psql/common.c | 51 ++++++--
src/test/regress/expected/psql_pipeline.out | 124 ++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 63 ++++++++++
3 files changed, 227 insertions(+), 11 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..c021eb97552a 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,19 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself. Such
+ * errors are not counted in available_results because they are
+ * not associated with a piped command. In that case, skip the
+ * counter decrements and continue to find the Sync result.
+ */
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2175,14 +2183,35 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by a Sync processing itself can leave some
+ * of them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ {
+ if (!ConnectionUp())
+ break;
+ continue;
+ }
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..a931d63cafe7 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,129 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+ ?column?
+----------
+ partial
+(1 row)
+
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+\endpipeline
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..468ef1d090b6 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,68 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+\getresults
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
[application/pgp-signature] signature.asc (833B, 3-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-05-31 09:00 Alexander Lakhin <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Alexander Lakhin @ 2026-05-31 09:00 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: [email protected]
Hello Michael,
29.05.2026 08:00, Michael Paquier wrote:
> When ending the pipeline the loop consuming the results is stuck, so
> we could check the connection state. We are going to enter in a
> freeze of the branches due to beta1 next week, so let's take our time.
>
> Please feel to use the v2 attached for your tests. I am also testing
> it more on my side.
Thank you for the fix! I haven't discovered new issues so far.
I've found a way to trigger another assertion, but I don't think it's
legitimate:
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -880,7 +880,7 @@ RemoveSocketFiles(void)
static void
socket_set_nonblocking(bool nonblocking)
{
- if (MyProcPort == NULL)
+ if ((MyProcPort == NULL) || (rand() % 10 == 0))
ereport(ERROR,
(errcode(ERRCODE_CONNECTION_DOES_NOT_EXIST),
errmsg("there is no client connection")));
makes this script:
(
echo "\startpipeline"
for i in {1..50}; do echo "\syncpipeline"; done
echo "
SELECT 1;
\endpipeline
\startpipeline
SELECT 2;
\endpipeline
"
) | psql
trigger
psql: common.c:2055: ExecQueryAndProcessResults: Assertion `pset.piped_syncs > 0' failed.
Probably there could be another way to throw an ERROR on \syncpipeline,
but I have no good idea yet.
Running psql_pipeline in a loop with the above modification applied:
for i in {1..1000}; do echo "ITERATION $i"; NO_TEMP_INSTALL=1 TESTS=psql_pipeline make -s check-tests; done
I also observed the test hanging (at iterations 284. 543, 218) due to loss
of synchronization between psql and postgres.
Best regards,
Alexander
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-06-01 02:11 Michael Paquier <[email protected]>
parent: Alexander Lakhin <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: Michael Paquier @ 2026-06-01 02:11 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: [email protected]
On Sun, May 31, 2026 at 12:00:01PM +0300, Alexander Lakhin wrote:
> I've found a way to trigger another assertion, but I don't think it's
> legitimate:
> --- a/src/backend/libpq/pqcomm.c
> +++ b/src/backend/libpq/pqcomm.c
> @@ -880,7 +880,7 @@ RemoveSocketFiles(void)
> static void
> socket_set_nonblocking(bool nonblocking)
> {
> - if (MyProcPort == NULL)
> + if ((MyProcPort == NULL) || (rand() % 10 == 0))
> ereport(ERROR,
> (errcode(ERRCODE_CONNECTION_DOES_NOT_EXIST),
> errmsg("there is no client connection")));
Server-side error injection. Noice.
> trigger
> psql: common.c:2055: ExecQueryAndProcessResults: Assertion
> `pset.piped_syncs > 0' failed.
This one would be in the same spirit as the others, if we cannot
really guarantee that the counters will be correct all the time we can
just more more defensive. That's an error thrown while the sync
message is processing itself, causing piped_syncs to get out of step.
> Probably there could be another way to throw an ERROR on \syncpipeline,
> but I have no good idea yet.
There is one challenge here, as far as I can see: libpq does not
really offer a way to make the difference between this thrown error
and an error that comes from a Sync, so it seems like we cannot do
much on the psql side except be more defensive? I am not sure if this
is worth the extra facility in libpq, the point would be moot in the
back branches anyway. And there is a benefit in keeping the psql code
as simple as possible, as well, so I'd tend to keep it more useful
still simpler.
> Running psql_pipeline in a loop with the above modification applied:
> for i in {1..1000}; do echo "ITERATION $i"; NO_TEMP_INSTALL=1 TESTS=psql_pipeline make -s check-tests; done
> I also observed the test hanging (at iterations 284. 543, 218) due to loss
> of synchronization between psql and postgres.
I have looked at that as well, and I don't think that this is fixable
only from the point of psql, because the error injected creates a
state where libpq's internal command queue gets out of sync regarding
what the backend has sent. The only thing that could be done is
inside libpq, as far as I can see, where we should try to detect that
the state is not synchronized anymore and fail rather than block. So
IMO, and with the error injected (which would never happen in
production in practice), the best thing I can come up with is the
attached for now.
One thing that I could see ourselves do as an extra improvement in
ExecQueryAndProcessResults() where we consume the results and check if
we're still in a busy state (some PQconsumeInput+PQisBusy). I don't
think that this should be a problem in practice, but this feels like
just hiding the real problem on the libpq side with the inconsistent
protocol state generated by the backend. I have also quickly tested
an approach based on that, unfortunately this leads to some
instability in the tests to due the async nature of the commands.
Anyway, the v3 attached passes the regression tests, handles the
pg_terminate_backend() case gracefully, handles the error case with
the error injected on backend-side a but better, and can avoid
some of the issues in the fourth case, but not all as we don't have
access to the pipe state when reaching the results do to the backend
missing up with the libpq state. Handling the 4th case more
gracefully would require some libpq changes, which may not justify the
cases we are dealing with here, at least to me. As a whole, I'd feel
that v3 is a good improvement in itself, and it addresses your
original issues and the assertions.
What do you think?
--
Michael
From 7fe4b3adf207f08d7a7bac2bc25521096959aa5e Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Mon, 1 Jun 2026 10:58:09 +0900
Subject: [PATCH v3] psql: Fix issues with deferred errors in pipelines
When an error is raised while processing a Sync message in a pipeline, e
like a deferred constraint violation, the error was not associated with
the piped command and was not counted in available_results. This caused
assertion failures in discardAbortedPipelineResults(), keeping an
incorrect state at pipeline exit, because the code assumed that
the number of available and requested results would always be positive,
expecting all the counters to be 0 at the end of a pipeline.
This commit switches discardAbortedPipelineResults() and
ExecQueryAndProcessResults() to take a softer approach when consuming
and draining the results after an error.
The reporter has shown a couple of assertion failures reachable. While
investigating more this issue I have bumped into two more. All these
cases are covered by the regression tests added in this commit, plus
some bonuses.
Reported-by: Alexander Lakhin <[email protected]>
Author: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 18
---
src/bin/psql/common.c | 76 +++++++++---
src/test/regress/expected/psql_pipeline.out | 124 ++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 63 ++++++++++
3 files changed, 245 insertions(+), 18 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..13202a974d1e 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,24 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself. Such
+ * errors are not counted in available_results because they are
+ * not associated with a piped command. In that case, skip the
+ * counter decrements and continue to find the Sync result.
+ *
+ * If the connection has been lost, there will never be any more
+ * results to read, so bail out.
+ */
+ if (!ConnectionUp())
+ return NULL;
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2044,14 +2057,16 @@ ExecQueryAndProcessResults(const char *query,
if (result_status == PGRES_PIPELINE_SYNC)
{
- Assert(pset.piped_syncs > 0);
-
/*
* Sync response, decrease the sync and requested_results
- * counters.
+ * counters. Guard against underflow: an error during Sync
+ * processing on the server can cause the client-side counter
+ * to drift.
*/
- pset.piped_syncs--;
- pset.requested_results--;
+ if (pset.piped_syncs > 0)
+ pset.piped_syncs--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
/*
* After a synchronisation point, reset success state to print
@@ -2073,8 +2088,10 @@ ExecQueryAndProcessResults(const char *query,
* In a pipeline with a non-sync response? Decrease the result
* counters.
*/
- pset.available_results--;
- pset.requested_results--;
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
/*
@@ -2175,14 +2192,37 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by a Sync processing itself can leave some
+ * of them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining;
+
+ remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ {
+ if (!ConnectionUp())
+ break;
+ continue;
+ }
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..a931d63cafe7 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,129 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+ ?column?
+----------
+ partial
+(1 row)
+
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+\endpipeline
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..468ef1d090b6 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,68 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+\getresults
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
Attachments:
[text/plain] v3-0001-psql-Fix-issues-with-deferred-errors-in-pipelines.patch (11.5K, 2-v3-0001-psql-Fix-issues-with-deferred-errors-in-pipelines.patch)
download | inline diff:
From 7fe4b3adf207f08d7a7bac2bc25521096959aa5e Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Mon, 1 Jun 2026 10:58:09 +0900
Subject: [PATCH v3] psql: Fix issues with deferred errors in pipelines
When an error is raised while processing a Sync message in a pipeline, e
like a deferred constraint violation, the error was not associated with
the piped command and was not counted in available_results. This caused
assertion failures in discardAbortedPipelineResults(), keeping an
incorrect state at pipeline exit, because the code assumed that
the number of available and requested results would always be positive,
expecting all the counters to be 0 at the end of a pipeline.
This commit switches discardAbortedPipelineResults() and
ExecQueryAndProcessResults() to take a softer approach when consuming
and draining the results after an error.
The reporter has shown a couple of assertion failures reachable. While
investigating more this issue I have bumped into two more. All these
cases are covered by the regression tests added in this commit, plus
some bonuses.
Reported-by: Alexander Lakhin <[email protected]>
Author: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 18
---
src/bin/psql/common.c | 76 +++++++++---
src/test/regress/expected/psql_pipeline.out | 124 ++++++++++++++++++++
src/test/regress/sql/psql_pipeline.sql | 63 ++++++++++
3 files changed, 245 insertions(+), 18 deletions(-)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 1a4e2ea0da82..13202a974d1e 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1499,11 +1499,24 @@ discardAbortedPipelineResults(void)
}
else if (res == NULL)
{
- /* A query was processed, decrement the counters */
- Assert(pset.available_results > 0);
- Assert(pset.requested_results > 0);
- pset.available_results--;
- pset.requested_results--;
+ /*
+ * A query was processed, decrement the counters.
+ *
+ * It is possible to get here with available_results == 0 when an
+ * error is generated by the Sync message processing itself. Such
+ * errors are not counted in available_results because they are
+ * not associated with a piped command. In that case, skip the
+ * counter decrements and continue to find the Sync result.
+ *
+ * If the connection has been lost, there will never be any more
+ * results to read, so bail out.
+ */
+ if (!ConnectionUp())
+ return NULL;
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
if (pset.requested_results == 0)
@@ -2044,14 +2057,16 @@ ExecQueryAndProcessResults(const char *query,
if (result_status == PGRES_PIPELINE_SYNC)
{
- Assert(pset.piped_syncs > 0);
-
/*
* Sync response, decrease the sync and requested_results
- * counters.
+ * counters. Guard against underflow: an error during Sync
+ * processing on the server can cause the client-side counter
+ * to drift.
*/
- pset.piped_syncs--;
- pset.requested_results--;
+ if (pset.piped_syncs > 0)
+ pset.piped_syncs--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
/*
* After a synchronisation point, reset success state to print
@@ -2073,8 +2088,10 @@ ExecQueryAndProcessResults(const char *query,
* In a pipeline with a non-sync response? Decrease the result
* counters.
*/
- pset.available_results--;
- pset.requested_results--;
+ if (pset.available_results > 0)
+ pset.available_results--;
+ if (pset.requested_results > 0)
+ pset.requested_results--;
}
/*
@@ -2175,14 +2192,37 @@ ExecQueryAndProcessResults(const char *query,
if (end_pipeline)
{
- /* after a pipeline is processed, pipeline piped_syncs should be 0 */
- Assert(pset.piped_syncs == 0);
- /* all commands have been processed */
- Assert(pset.piped_commands == 0);
- /* all results were read */
- Assert(pset.available_results == 0);
+ /*
+ * Reset available/requested results. Normally these are already 0,
+ * but an error generated by a Sync processing itself can leave some
+ * of them behind. Consume them before exiting pipeline mode.
+ */
+ while (pset.piped_syncs > 0)
+ {
+ PGresult *remaining;
+
+ remaining = PQgetResult(pset.db);
+
+ if (remaining == NULL)
+ {
+ if (!ConnectionUp())
+ break;
+ continue;
+ }
+ if (PQresultStatus(remaining) == PGRES_PIPELINE_SYNC)
+ pset.piped_syncs--;
+ PQclear(remaining);
+ }
+ pset.piped_syncs = 0;
+ pset.piped_commands = 0;
+ pset.available_results = 0;
+ pset.requested_results = 0;
+
+ if (PQpipelineStatus(pset.db) != PQ_PIPELINE_OFF)
+ PQexitPipelineMode(pset.db);
}
Assert(pset.requested_results == 0);
+
SetPipelineVariables();
/* may need this to recover from conn loss during COPY */
diff --git a/src/test/regress/expected/psql_pipeline.out b/src/test/regress/expected/psql_pipeline.out
index a0816fb10b68..a931d63cafe7 100644
--- a/src/test/regress/expected/psql_pipeline.out
+++ b/src/test/regress/expected/psql_pipeline.out
@@ -764,5 +764,129 @@ VACUUM psql_pipeline \bind \sendpipeline
1
(1 row)
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+ a
+---
+ 1
+ 1
+(2 rows)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+ ?column?
+--------------
+ after_sync_1
+(1 row)
+
+ ?column?
+--------------
+ after_sync_2
+(1 row)
+
+ ?column?
+--------------
+ after_sync_3
+(1 row)
+
+ ?column?
+--------------
+ after_sync_4
+(1 row)
+
+ ?column?
+--------------
+ after_sync_5
+(1 row)
+
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+ ?column?
+----------
+ partial
+(1 row)
+
+\getresults
+ERROR: duplicate key value violates unique constraint "psql_pipeline_defer_pkey"
+DETAIL: Key (a)=(1) already exists.
+\endpipeline
+DROP TABLE psql_pipeline_defer;
-- Clean up
DROP TABLE psql_pipeline;
diff --git a/src/test/regress/sql/psql_pipeline.sql b/src/test/regress/sql/psql_pipeline.sql
index 6788dceee2e9..468ef1d090b6 100644
--- a/src/test/regress/sql/psql_pipeline.sql
+++ b/src/test/regress/sql/psql_pipeline.sql
@@ -438,5 +438,68 @@ SELECT 1 \bind \sendpipeline
VACUUM psql_pipeline \bind \sendpipeline
\endpipeline
+-- Deferred constraint violation at commit time in a pipeline.
+CREATE TABLE psql_pipeline_defer (a INTEGER PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) RETURNING * \bind 1 \sendpipeline
+\endpipeline
+
+-- Same with \syncpipeline and commands after the failing sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\endpipeline
+
+-- More patterns with more \syncpipeline, more commands and \getresults
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+\endpipeline
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+\getresults
+SELECT $1 \bind 'after_sync_1' \sendpipeline
+\getresults
+SELECT $1 \bind 'after_sync_2' \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'after_sync_3' \sendpipeline
+SELECT $1 \bind 'after_sync_4' \sendpipeline
+SELECT $1 \bind 'after_sync_5' \sendpipeline
+\endpipeline
+
+-- Deferred error combined with a regular command error after the sync.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\syncpipeline
+SELECT $1 \bind \sendpipeline
+SELECT $1 \bind 'after_error' \sendpipeline
+\endpipeline
+
+-- Empty sync segment followed by a deferred error.
+\startpipeline
+\syncpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+\endpipeline
+
+-- Deferred error with \getresults reading results one at a time.
+\startpipeline
+INSERT INTO psql_pipeline_defer VALUES ($1), ($1) \bind 1 \sendpipeline
+SELECT $1 \bind 'partial' \sendpipeline
+\syncpipeline
+\getresults 1
+\getresults 1
+\getresults
+\endpipeline
+
+DROP TABLE psql_pipeline_defer;
+
-- Clean up
DROP TABLE psql_pipeline;
--
2.54.0
[application/pgp-signature] signature.asc (833B, 3-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-06-01 05:00 Alexander Lakhin <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 2 replies; 10+ messages in thread
From: Alexander Lakhin @ 2026-06-01 05:00 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: [email protected]
Hello Michael,
01.06.2026 05:11, Michael Paquier пишет:
> I have looked at that as well, and I don't think that this is fixable
> only from the point of psql, because the error injected creates a
> state where libpq's internal command queue gets out of sync regarding
> what the backend has sent. The only thing that could be done is
> inside libpq, as far as I can see, where we should try to detect that
> the state is not synchronized anymore and fail rather than block. So
> IMO, and with the error injected (which would never happen in
> production in practice), the best thing I can come up with is the
> attached for now.
>
> One thing that I could see ourselves do as an extra improvement in
> ExecQueryAndProcessResults() where we consume the results and check if
> we're still in a busy state (some PQconsumeInput+PQisBusy). I don't
> think that this should be a problem in practice, but this feels like
> just hiding the real problem on the libpq side with the inconsistent
> protocol state generated by the backend. I have also quickly tested
> an approach based on that, unfortunately this leads to some
> instability in the tests to due the async nature of the commands.
>
> Anyway, the v3 attached passes the regression tests, handles the
> pg_terminate_backend() case gracefully, handles the error case with
> the error injected on backend-side a but better, and can avoid
> some of the issues in the fourth case, but not all as we don't have
> access to the pipe state when reaching the results do to the backend
> missing up with the libpq state. Handling the 4th case more
> gracefully would require some libpq changes, which may not justify the
> cases we are dealing with here, at least to me. As a whole, I'd feel
> that v3 is a good improvement in itself, and it addresses your
> original issues and the assertions.
>
> What do you think?
I agree with your points, the v3 looks good to me. Thank you for paying
attention to all of these issues!
Best regards,
Alexander
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-06-01 06:18 Michael Paquier <[email protected]>
parent: Alexander Lakhin <[email protected]>
1 sibling, 0 replies; 10+ messages in thread
From: Michael Paquier @ 2026-06-01 06:18 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: [email protected]
On Mon, Jun 01, 2026 at 08:00:01AM +0300, Alexander Lakhin wrote:
> I agree with your points, the v3 looks good to me. Thank you for paying
> attention to all of these issues!
Okay, thanks!
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert
@ 2026-06-02 23:59 Michael Paquier <[email protected]>
parent: Alexander Lakhin <[email protected]>
1 sibling, 0 replies; 10+ messages in thread
From: Michael Paquier @ 2026-06-02 23:59 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: [email protected]
On Mon, Jun 01, 2026 at 08:00:01AM +0300, Alexander Lakhin wrote:
> I agree with your points, the v3 looks good to me. Thank you for paying
> attention to all of these issues!
And now applied down to v18 as of d21604e17e49. Thanks, Alexander.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, 2-signature.asc)
download
^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2026-06-02 23:59 UTC | newest]
Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-05-26 19:00 BUG #19494: Error on transaction commit inside pipeline triggers psql's Assert PG Bug reporting form <[email protected]>
2026-05-28 03:51 ` Michael Paquier <[email protected]>
2026-05-28 05:26 ` Michael Paquier <[email protected]>
2026-05-29 04:00 ` Alexander Lakhin <[email protected]>
2026-05-29 05:00 ` Michael Paquier <[email protected]>
2026-05-31 09:00 ` Alexander Lakhin <[email protected]>
2026-06-01 02:11 ` Michael Paquier <[email protected]>
2026-06-01 05:00 ` Alexander Lakhin <[email protected]>
2026-06-01 06:18 ` Michael Paquier <[email protected]>
2026-06-02 23:59 ` Michael Paquier <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox