public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 3/9] Optimize allocations in bringetbitmap
34+ messages / 12 participants
[nested] [flat]
* [PATCH 3/9] Optimize allocations in bringetbitmap
@ 2020-09-13 10:12 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 34+ messages in thread
From: Tomas Vondra @ 2020-09-13 10:12 UTC (permalink / raw)
The bringetbitmap function allocates memory for various purposes, which
may be quite expensive, depending on the number of scan keys. Instead of
allocating them separately, allocate one bit chunk of memory an carve it
into smaller pieces as needed - all the pieces have the same lifespan,
and it saves quite a bit of CPU and memory overhead.
Author: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/access/brin/brin.c | 62 +++++++++++++++++++++++++++-------
1 file changed, 49 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 14da9ed17f..3735c41788 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -373,6 +373,9 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
int *nkeys,
*nnullkeys;
int keyno;
+ char *ptr;
+ Size len;
+ char *tmp PG_USED_FOR_ASSERTS_ONLY;
opaque = (BrinOpaque *) scan->opaque;
bdesc = opaque->bo_bdesc;
@@ -398,11 +401,50 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
* Make room for per-attribute lists of scan keys that we'll pass to the
* consistent support procedure. We keep null and regular keys separate,
* so that we can easily pass regular keys to the consistent function.
+ *
+ * To reduce the allocation overhead, we allocate one big chunk and then
+ * carve it into smaller arrays ourselves. All the pieces have exactly
+ * the same lifetime, so that's OK.
*/
- keys = palloc0(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
- nullkeys = palloc0(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
- nkeys = palloc0(sizeof(int) * bdesc->bd_tupdesc->natts);
- nnullkeys = palloc0(sizeof(int) * bdesc->bd_tupdesc->natts);
+ len =
+ /* regular keys */
+ MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts) +
+ /* NULL keys */
+ MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ ptr = palloc(len);
+ tmp = ptr;
+
+ keys = (ScanKey **) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
+
+ nullkeys = (ScanKey **) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
+
+ nkeys = (int *) ptr;
+ ptr += MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ nnullkeys = (int *) ptr;
+ ptr += MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ for (int i = 0; i < bdesc->bd_tupdesc->natts; i++)
+ {
+ keys[i] = (ScanKey *) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys);
+
+ nullkeys[i] = (ScanKey *) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys);
+ }
+
+ Assert(tmp + len == ptr);
+
+ /* zero the number of keys */
+ memset(nkeys, 0, sizeof(int) * bdesc->bd_tupdesc->natts);
+ memset(nnullkeys, 0, sizeof(int) * bdesc->bd_tupdesc->natts);
/*
* Preprocess the scan keys - split them into per-attribute arrays.
@@ -438,9 +480,9 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
{
FmgrInfo *tmp;
- /* No key/null arrays for this attribute. */
- Assert((keys[keyattno - 1] == NULL) && (nkeys[keyattno - 1] == 0));
- Assert((nullkeys[keyattno - 1] == NULL) && (nnullkeys[keyattno - 1] == 0));
+ /* First time we see this attribute, so no key/null keys. */
+ Assert(nkeys[keyattno - 1] == 0);
+ Assert(nnullkeys[keyattno - 1] == 0);
tmp = index_getprocinfo(idxRel, keyattno,
BRIN_PROCNUM_CONSISTENT);
@@ -451,17 +493,11 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
/* Add key to the proper per-attribute array. */
if (key->sk_flags & SK_ISNULL)
{
- if (!nullkeys[keyattno - 1])
- nullkeys[keyattno - 1] = palloc0(sizeof(ScanKey) * scan->numberOfKeys);
-
nullkeys[keyattno - 1][nnullkeys[keyattno - 1]] = key;
nnullkeys[keyattno - 1]++;
}
else
{
- if (!keys[keyattno - 1])
- keys[keyattno - 1] = palloc0(sizeof(ScanKey) * scan->numberOfKeys);
-
keys[keyattno - 1][nkeys[keyattno - 1]] = key;
nkeys[keyattno - 1]++;
}
--
2.26.2
--------------4B194FF8F3EA3786FF9EAE1F
Content-Type: text/x-patch; charset=UTF-8;
name="0002-Move-IS-NOT-NULL-handling-from-BRIN-support-20210203.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="0002-Move-IS-NOT-NULL-handling-from-BRIN-support-20210203.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-24 21:41 Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Tom Lane @ 2022-03-24 21:41 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; +Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected]
Jacob Champion <[email protected]> writes:
> On Thu, 2022-01-13 at 14:51 +0000, Jelte Fennema wrote:
>> 2. Cancel connections benefit automatically from any improvements made
>> to the normal connection establishment codepath. Examples of things
>> that it currently gets for free currently are TLS support and
>> keepalive settings.
> This seems like a big change compared to PQcancel(); one that's not
> really hinted at elsewhere. Having the async version of an API open up
> a completely different code path with new features is pretty surprising
> to me.
Well, the patch lacks any user-facing doco at all, so a-fortiori this
point is not covered. I trust the plan was to write docs later.
I kind of feel that this patch is going in the wrong direction.
I do see the need for a version of PQcancel that can encrypt the
transmitted cancel request (and yes, that should work on the backend
side; see recursion in ProcessStartupPacket). I have not seen
requests for a non-blocking version, and this doesn't surprise me.
I feel that the whole non-blocking aspect of libpq probably belongs
to another era when people didn't trust threads.
So what I'd do is make a version that just takes a PGconn, sends the
cancel request, and returns success or failure; never mind the
non-blocking aspect. One possible long-run advantage of this is that
it might be possible to "sync" the cancel request so that we know,
or at least can find out afterwards, exactly which query got
cancelled; something that's fundamentally impossible if the cancel
function works from a clone data structure that is disconnected
from the current connection state.
(Note that it probably makes sense to make a clone PGconn to pass
to fe-connect.c, internally to this function. I just don't want
to expose that to the app.)
regards, tom lane
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
@ 2022-03-24 22:49 ` Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Andres Freund @ 2022-03-24 22:49 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jacob Champion <[email protected]>; [email protected] <[email protected]>; [email protected]
Hi,
On 2022-03-24 17:41:53 -0400, Tom Lane wrote:
> I kind of feel that this patch is going in the wrong direction.
> I do see the need for a version of PQcancel that can encrypt the
> transmitted cancel request (and yes, that should work on the backend
> side; see recursion in ProcessStartupPacket). I have not seen
> requests for a non-blocking version, and this doesn't surprise me.
> I feel that the whole non-blocking aspect of libpq probably belongs
> to another era when people didn't trust threads.
That's not a whole lot of fun if you think of cases like postgres_fdw (or
citus as in Jelte's case), which run inside the backend. Even with just a
single postgres_fdw, we don't really want to end up in an uninterruptible
PQcancel() that doesn't even react to pg_terminate_backend().
Even if using threads weren't an issue, I don't really buy the premise - most
networking code has moved *away* from using dedicated threads for each
connection. It just doesn't scale.
Leaving PQcancel aside, we use the non-blocking libpq stuff widely
ourselves. I think walreceiver, isolationtester, pgbench etc would be *much*
harder to get working equally well if there was just blocking calls. If
anything, we're getting to the point where purely blocking functionality
shouldn't be added anymore.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
@ 2022-03-25 18:34 ` Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Robert Haas @ 2022-03-25 18:34 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 24, 2022 at 6:49 PM Andres Freund <[email protected]> wrote:
> That's not a whole lot of fun if you think of cases like postgres_fdw (or
> citus as in Jelte's case), which run inside the backend. Even with just a
> single postgres_fdw, we don't really want to end up in an uninterruptible
> PQcancel() that doesn't even react to pg_terminate_backend().
>
> Even if using threads weren't an issue, I don't really buy the premise - most
> networking code has moved *away* from using dedicated threads for each
> connection. It just doesn't scale.
>
> Leaving PQcancel aside, we use the non-blocking libpq stuff widely
> ourselves. I think walreceiver, isolationtester, pgbench etc would be *much*
> harder to get working equally well if there was just blocking calls. If
> anything, we're getting to the point where purely blocking functionality
> shouldn't be added anymore.
+1. I think having a non-blocking version of PQcancel() available is a
great idea, and I've wanted it myself. See commit
ae9bfc5d65123aaa0d1cca9988037489760bdeae.
That said, I don't think that this particular patch is going in the
right direction. I think Jacob's comment upthread is right on point:
"This seems like a big change compared to PQcancel(); one that's not
really hinted at elsewhere. Having the async version of an API open up
a completely different code path with new features is pretty
surprising to me." It seems to me that we want to end up with similar
code paths for PQcancel() and the non-blocking version of cancel. We
could get there in two ways. One way would be to implement the
non-blocking functionality in a manner that matches exactly what
PQcancel() does now. I imagine that the existing code from PQcancel()
would move, with some amount of change, into a new set of non-blocking
APIs. Perhaps PQcancel() would then be rewritten to use those new APIs
instead of hand-rolling the same logic. The other possible approach
would be to first change the blocking version of PQcancel() to use the
regular connection code instead of its own idiosyncratic logic, and
then as a second step, extend it with non-blocking interfaces that use
the regular non-blocking connection code. With either of these
approaches, we end up with the functionality working similarly in the
blocking and non-blocking code paths.
Leaving the question of approach aside, I think it's fairly clear that
this patch cannot be seriously considered for v15. One problem is the
lack of user-facing documentation, but there's a other stuff that just
doesn't look sufficiently well-considered. For example, it updates the
comment for pqsecure_read() to say "Returns -1 in case of failures,
except in the case of clean connection closure then it returns -2."
But that function calls any of three different implementation
functions depending on the situation and the patch only updates one of
them. And it updates that function to return -2 when the is
ECONNRESET, which seems to fly in the face of the comment's idea that
this is the "clean connection closure" case. I think it's probably a
bad sign that this function is tinkering with logic in this sort of
low-level function anyway. pqReadData() is a really general function
that manages to work with non-blocking I/O already, so why does
non-blocking query cancellation need to change its return values, or
whether or not it drops data in certain cases?
I'm also skeptical about the fact that we end up with a whole bunch of
new functions that are just wrappers around existing functions. That's
not a scalable approach. Every function that we have for a PGconn will
eventually need a variant that deals with a PGcancelConn. That seems
kind of pointless, especially considering that a PGcancelConn is
*exactly* a PGconn in disguise. If we decide to pursue the approach of
using the existing infrastructure for PGconn objects to handle query
cancellation, we ought to manipulate them using the same functions we
currently do, with some kind of mode or flag or switch or something
that you can use to turn a regular PGconn into something that cancels
a query. Maybe you create the PGconn and call
PQsprinkleMagicCancelDust() on it, and then you just proceed using the
existing functions, or something like that. Then, not only do the
existing functions not need query-cancel analogues, but any new
functions we add in the future don't either.
I'll set the target version for this patch to 16. I hope work continues.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
@ 2022-03-25 18:46 ` Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Tom Lane @ 2022-03-25 18:46 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
Robert Haas <[email protected]> writes:
> That said, I don't think that this particular patch is going in the
> right direction. I think Jacob's comment upthread is right on point:
> "This seems like a big change compared to PQcancel(); one that's not
> really hinted at elsewhere. Having the async version of an API open up
> a completely different code path with new features is pretty
> surprising to me." It seems to me that we want to end up with similar
> code paths for PQcancel() and the non-blocking version of cancel. We
> could get there in two ways. One way would be to implement the
> non-blocking functionality in a manner that matches exactly what
> PQcancel() does now. I imagine that the existing code from PQcancel()
> would move, with some amount of change, into a new set of non-blocking
> APIs. Perhaps PQcancel() would then be rewritten to use those new APIs
> instead of hand-rolling the same logic. The other possible approach
> would be to first change the blocking version of PQcancel() to use the
> regular connection code instead of its own idiosyncratic logic, and
> then as a second step, extend it with non-blocking interfaces that use
> the regular non-blocking connection code. With either of these
> approaches, we end up with the functionality working similarly in the
> blocking and non-blocking code paths.
I think you misunderstand where the real pain point is. The reason
that PQcancel's functionality is so limited has little to do with
blocking vs non-blocking, and everything to do with the fact that
it's designed to be safe to call from a SIGINT handler. That makes
it quite impractical to invoke OpenSSL, and probably our GSS code
as well. If we want support for all connection-time options then
we have to make a new function that does not promise signal safety.
I'm prepared to yield on the question of whether we should provide
a non-blocking version, though I still say that (a) an easier-to-call,
one-step blocking alternative would be good too, and (b) it should
not be designed around the assumption that there's a completely
independent state object being used to perform the cancel. Even in
the non-blocking case, callers should only deal with the original
PGconn.
> Leaving the question of approach aside, I think it's fairly clear that
> this patch cannot be seriously considered for v15.
Yeah, I don't think it's anywhere near fully baked yet. On the other
hand, we do have a couple of weeks left.
regards, tom lane
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
@ 2022-03-25 19:22 ` Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Robert Haas @ 2022-03-25 19:22 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 25, 2022 at 2:47 PM Tom Lane <[email protected]> wrote:
> I think you misunderstand where the real pain point is. The reason
> that PQcancel's functionality is so limited has little to do with
> blocking vs non-blocking, and everything to do with the fact that
> it's designed to be safe to call from a SIGINT handler. That makes
> it quite impractical to invoke OpenSSL, and probably our GSS code
> as well. If we want support for all connection-time options then
> we have to make a new function that does not promise signal safety.
Well, that's a fair point, but it's somewhat orthogonal to the one I'm
making, which is that a non-blocking version of function X might be
expected to share code or at least functionality with X itself. Having
something that is named in a way that implies asynchrony without other
differences but which is actually different in other important ways is
no good.
> I'm prepared to yield on the question of whether we should provide
> a non-blocking version, though I still say that (a) an easier-to-call,
> one-step blocking alternative would be good too, and (b) it should
> not be designed around the assumption that there's a completely
> independent state object being used to perform the cancel. Even in
> the non-blocking case, callers should only deal with the original
> PGconn.
Well, this sounds like you're arguing for the first of the two
approaches I thought would be acceptable, rather than the second.
> > Leaving the question of approach aside, I think it's fairly clear that
> > this patch cannot be seriously considered for v15.
>
> Yeah, I don't think it's anywhere near fully baked yet. On the other
> hand, we do have a couple of weeks left.
We do?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
@ 2022-03-25 19:34 ` Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Tom Lane @ 2022-03-25 19:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
Robert Haas <[email protected]> writes:
> Well, that's a fair point, but it's somewhat orthogonal to the one I'm
> making, which is that a non-blocking version of function X might be
> expected to share code or at least functionality with X itself. Having
> something that is named in a way that implies asynchrony without other
> differences but which is actually different in other important ways is
> no good.
Yeah. We need to choose a name for these new function(s) that is
sufficiently different from "PQcancel" that people won't expect them
to behave exactly the same as that does. I lack any good ideas about
that, how about you?
>> Yeah, I don't think it's anywhere near fully baked yet. On the other
>> hand, we do have a couple of weeks left.
> We do?
Um, you did read the psql-release discussion about setting the feature
freeze deadline, no?
regards, tom lane
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
@ 2022-03-28 09:28 ` Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-03-28 09:28 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; PostgreSQL Hackers <[email protected]>
Thanks for all the feedback everyone. I'll try to send a new patch
later this week that includes user facing docs and a simplified API.
For now a few responses:
> Yeah. We need to choose a name for these new function(s) that is
> sufficiently different from "PQcancel" that people won't expect them
> to behave exactly the same as that does. I lack any good ideas about
> that, how about you?
So I guess the names I proposed were not great, since everyone seems to be falling over them.
But I'd like to make my intention clear with the current naming. After this patch there would be
four different APIs for starting a cancelation:
1. PQrequestCancel: deprecated+old, not signal-safe function for requesting query cancellation, only uses a specific set of connection options
2. PQcancel: Cancel queries in a signal safe way, to be signal-safe it only uses a limited set of connection options
3. PQcancelConnect: Cancel queries in a non-signal safe way that uses all connection options
4. PQcancelConnectStart: Cancel queries in a non-signal safe and non-blocking way that uses all connection options
So the idea was that you should not look at PQcancelConnectStart as the non-blocking
version of PQcancel, but as the non-blocking version of PQcancelConnect. I'll try to
think of some different names too, but IMHO these names could be acceptable
when their differences are addressed sufficiently in the documentation.
One other approach to naming that comes to mind now is repurposing PQrequestCancel:
1. PQrequestCancel: Cancel queries in a non-signal safe way that uses all connection options
2. PQrequestCancelStart: Cancel queries in a non-signal safe and non-blocking way that uses all connection options
3. PQcancel: Cancel queries in a signal safe way, to be signal-safe it only uses a limited set of connection options
> I think it's probably a
> bad sign that this function is tinkering with logic in this sort of
> low-level function anyway. pqReadData() is a really general function
> that manages to work with non-blocking I/O already, so why does
> non-blocking query cancellation need to change its return values, or
> whether or not it drops data in certain cases?
The reason for this low level change is that the cancellation part of the
Postgres protocol is following a different, much more simplistic design
than all the other parts. The client does not expect a response message back
from the server after sending the cancellation request. The expectation
is that the server signals completion by closing the connection, i.e. sending EOF.
For all other parts of the protocol, connection termination should be initiated
client side by sending a Terminate message. So the server closing (sending
EOF) is always unexpected and is thus currently considered an error by pqReadData.
But since this is not the case for the cancellation protocol, the result is
changed to -2 in case of EOF to make it possible to distinguish between
an EOF and an actual error.
> And it updates that function to return -2 when the is
> ECONNRESET, which seems to fly in the face of the comment's idea that
> this is the "clean connection closure" case.
The diff sadly does not include the very relevant comment right above these
lines. Pasting the whole case statement here to clear up this confusion:
case SSL_ERROR_ZERO_RETURN:
/*
* Per OpenSSL documentation, this error code is only returned for
* a clean connection closure, so we should not report it as a
* server crash.
*/
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
n = -2;
break;
> For example, it updates the
> comment for pqsecure_read() to say "Returns -1 in case of failures,
> except in the case of clean connection closure then it returns -2."
> But that function calls any of three different implementation
> functions depending on the situation and the patch only updates one of
> them.
That comment is indeed not describing what is happening correctly and I'll
try to make it clearer. The main reason for it being incorrect is coming from
the fact that receiving EOFs is handled in different places based on the
encryption method:
1. Unencrypted TCP: EOF is not returned as an error by pqsecure_read, but detected by pqReadData (see comments related to definitelyEOF)
2. OpenSSL: EOF is returned as an error by pqsecure_read (see copied case statement above)
3. GSS: When writing the patch I was not sure how EOF handling worked here, but given that the tests passed for Jacob on GSS, I'm guessing it works the same as unencrypted TCP.
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-03-30 16:08 ` Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-03-30 16:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; PostgreSQL Hackers <[email protected]>
I attached a new version of this patch. Which does three main things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe, but
blocking cancel (without having a need for signal safety).
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing documentation
for all these functions.
As a API design change from the previous version, PQrequestCancelStart now
returns a regular PGconn for the cancel connection.
@Tom Lane regarding this:
> Even in the non-blocking case, callers should only deal with the original PGconn.
This would by definition result in non-threadsafe code (afaict). So I refrained from doing this.
The blocking version doesn't expose a PGconn at all, but the non-blocking one now returns a new PGconn.
There's two more changes that I at least want to do before considering this patch mergable:
1. Go over all the functions that can be called with a PGconn, but should not be
called with a cancellation PGconn and error out or exit early.
2. Copy over the SockAddr from the original connection and always connect to
the same socket. I believe with the current code the cancellation could end up
at the wrong server if there are multiple hosts listed in the connection string.
And there's a third item that I would like to do as a bonus:
3. Actually use the non-blocking API for the postgres_fdw code to implement a
timeout. Which would allow this comment can be removed:
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
* amount of time that we might block inside PQgetCancel().
*/
So a next version of this patch can be expected somewhere later this week.
But any feedback on the current version would be appreciated. Because
these 3 changes won't change the overall design much.
Jelte
Attachments:
[application/octet-stream] 0001-Add-documentation-for-libpq_pipeline-tests.patch (1.5K, ../../HE1PR8303MB00735DC38F45BD63A3A49347F71F9@HE1PR8303MB0073.EURPRD83.prod.outlook.com/2-0001-Add-documentation-for-libpq_pipeline-tests.patch)
download | inline diff:
From 22a02899d47d46ed05ada2e38e3f9804981b96eb Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Thu, 13 Jan 2022 15:26:35 +0100
Subject: [PATCH 1/2] Add documentation for libpq_pipeline tests
This adds some explanation on how to run and add libpq tests.
---
src/test/modules/libpq_pipeline/README | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
index d8174dd579..6eda6c5756 100644
--- a/src/test/modules/libpq_pipeline/README
+++ b/src/test/modules/libpq_pipeline/README
@@ -1 +1,21 @@
Test programs and libraries for libpq
+=====================================
+
+You can manually run a specific test by running:
+
+ ./libpq_pipeline <name of test>
+
+To add a new libpq test to this module you need to edit libpq_pipeline.c. There
+you should add the name of your new test to the "print_test_list" function.
+Then in main you should do something when this test name is passed to the
+program.
+
+If the order in which postgres protocol messages are sent is deterministic for
+your test. Then you can generate a trace of these messages using the following
+command:
+
+ ./libpq_pipeline mynewtest -t traces/mynewtest.trace
+
+Once you've done that you should make sure that when running "make check"
+the generated trace is compared to the expected trace. This is done by adding
+your test name to the $cmptrace definition in the t/001_libpq_pipeline.pl file
--
2.17.1
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (39.0K, ../../HE1PR8303MB00735DC38F45BD63A3A49347F71F9@HE1PR8303MB0073.EURPRD83.prod.outlook.com/3-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 98a67a65eee6b7ee1275e48e5053ba8ab3055014 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does two things:
1. Change PQrequestCancel to use the regular connection establishement,
to address a few security issues.
2. Add PQrequestCancelStart which is a thread safe and non blocking
version of this new PQrequestCancel implementation.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQrequestCancelStart. This API can be used to send cancellations in a
non-blocking fashion.
This patch also includes a test for this all of libpq cancellation APIs.
The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 12 +-
contrib/postgres_fdw/connection.c | 11 +-
doc/src/sgml/libpq.sgml | 212 +++++++++++++----
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 225 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 13 +-
src/interfaces/libpq/libpq-int.h | 2 +
src/test/isolation/isolationtester.c | 29 +--
.../modules/libpq_pipeline/libpq_pipeline.c | 214 ++++++++++++++++-
13 files changed, 627 insertions(+), 126 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a06d4bd12d..30cbb22a22 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1380,22 +1380,14 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
-
- if (res == 1)
+ if (PQrequestCancel(conn))
PG_RETURN_TEXT_P(cstring_to_text("OK"));
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(PQerrorMessage(conn)));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..2e182645f7 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1263,8 +1263,6 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
TimestampTz endtime;
bool timed_out;
@@ -1279,19 +1277,14 @@ pgfdw_cancel_query(PGconn *conn)
* Issue cancel request. Unfortunately, there's no good way to limit the
* amount of time that we might block inside PQgetCancel().
*/
- if ((cancel = PQgetCancel(conn)))
- {
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ if (!PQrequestCancel(conn))
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
+ pchomp(PQerrorMessage(conn)))));
return false;
}
- PQfreeCancel(cancel);
- }
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 0b2a8720f0..6b0683b9b0 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8835,10 +8967,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index a30c66f13a..ff18dab043 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -162,19 +162,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index cf554d389f..5462e1305c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,11 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +741,58 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Copy over information needed to cancel
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +970,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2134,6 +2230,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2379,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2399,17 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+ if (n == -2 && conn->cancelRequest)
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2419,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2758,6 +2877,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3095,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4342,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4464,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4822,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4854,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
+ PGconn *cancelConn = NULL;
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index d3bf57b850..4ffaea63c1 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -252,7 +252,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..42367d4886 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -165,6 +168,10 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -282,6 +289,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +338,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel */
extern int PQrequestCancel(PGconn *conn);
+/* non blocking and thread safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..ff9555e263 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,8 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest;
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..fe1ca168c8 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,17 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
- {
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ if (PQrequestCancel(conn)) {
+ /*
+ * print to stdout not stderr, as this should appear
+ * in the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..95f1d5eb2f 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,215 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+static void
+confirm_query_cancelled(PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal("PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal("query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal("query failed with a different error than cancellation: %s", PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1754,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1852,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.17.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-03-31 05:47 ` Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Justin Pryzby @ 2022-03-31 05:47 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected]
Note that the patch is still variously failing in cirrus.
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/37/3511
You may already know that it's possible to trigger the cirrus ci tasks using a
github branch. See src/tools/ci/README.
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
@ 2022-04-01 16:13 ` Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-04-01 16:13 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>
Attached is the latest version of this patch, which I think is now in a state
in which it could be merged. The changes are:
1. Don't do host and address discovery for cancel connections. It now
reuses raddr and whichhost from the original connection. This makes
sure the cancel always goes to the right server, even when DNS records
change or another server would be chosen now in case of connnection
strings containing multiple hosts.
2. Fix the windows CI failure. This is done by both using the threadsafe code
in the the dblink cancellation code, and also by not erroring a cancellation
connection on windows in case of any errors. This last one is to work around
the issue described in this thread:
https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
I also went over most of the functions that take a PGconn, to see if they needed
extra checks to guard against being executed on cancel. So far all seemed fine,
either they should be okay to execute against a cancellation connection, or
they failed already anyway because a cancellation connection never reaches
the CONNECTION_OK state. So I didn't add any checks specifically for cancel
connections. I'll do this again next week with a fresh head, to see if I haven't
missed any cases.
I'll try to find some time early next week to implement non-blocking cancellation
usage in postgres_fdw, i.e. the bonus task I mentioned in my previous email. But
I don't think it's necessary to have that implemented before merging.
Attachments:
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (45.5K, ../../HE1PR8303MB00732A831BE871DF360A293CF7E09@HE1PR8303MB0073.EURPRD83.prod.outlook.com/2-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 8277c3f5eeaf13f018ab9b5899650f109c4a45b6 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does two things:
1. Change PQrequestCancel to use the regular connection establishement,
to address a few security issues.
2. Add PQrequestCancelStart which is a thread safe and non blocking
version of this new PQrequestCancel implementation.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQrequestCancelStart. This API can be used to send cancellations in a
non-blocking fashion.
This patch also includes a test for this all of libpq cancellation APIs.
The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 31 +-
contrib/postgres_fdw/connection.c | 19 +-
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 214 ++++++++++-
13 files changed, 743 insertions(+), 150 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a06d4bd12d..f3935331f6 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1380,22 +1380,35 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
+ {
+ PG_RETURN_TEXT_P(cstring_to_text("out of memory"));
+ }
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..8ad810d621 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1263,8 +1263,6 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
TimestampTz endtime;
bool timed_out;
@@ -1279,18 +1277,13 @@ pgfdw_cancel_query(PGconn *conn)
* Issue cancel request. Unfortunately, there's no good way to limit the
* amount of time that we might block inside PQgetCancel().
*/
- if ((cancel = PQgetCancel(conn)))
+ if (!PQrequestCancel(conn))
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
- {
- ereport(WARNING,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
- }
- PQfreeCancel(cancel);
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(conn)))));
+ return false;
}
/* Get and discard the result of the query. */
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 1c20901c3c..45f8001fbd 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8850,10 +8982,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index a30c66f13a..ff18dab043 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -162,19 +162,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index cf554d389f..e8356e75a2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 24a598b6e4..8a2a7c112c 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..50b6a7bc7d 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..4e53d3c165 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,215 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+static void
+confirm_query_cancelled(PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal("PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal("query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal("query failed with a different error than cancellation: %s", PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1754,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1852,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.17.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-04-04 15:21 ` Jelte Fennema <[email protected]>
2022-06-25 00:36 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 2 replies; 34+ messages in thread
From: Jelte Fennema @ 2022-04-04 15:21 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>
Hereby what I consider the final version of this patch. I don't have any
changes planned myself (except for ones that come up during review).
Things that changed since the previous iteration:
1. postgres_fdw now uses the non-blocking cancellation API (including test).
2. Added some extra sleeps to the cancellation test, to remove random failures on FreeBSD.
Attachments:
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (51.1K, ../../AM5PR83MB037283BB5E7BE0EEE0016E6BF7E59@AM5PR83MB0372.EURPRD83.prod.outlook.com/2-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From ebb611ca522a5fbabf9334ed47681e9490644aea Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does four things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe but
blocking cancel (without having a need for signal-safety).
4. Use this new cancellation API everywhere in the codebase where
signal-safety is not a necessity.
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing
documentation for all the newly added functions.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQrequestCancelStart can now be used
instead, to have a non-blocking way of sending cancel requests. The
postgres_fdw cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 28 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 229 +++++++++++-
15 files changed, 849 insertions(+), 153 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a06d4bd12d..551dc8617a 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1380,22 +1380,30 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..c270ac3dd1 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1263,35 +1263,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGconn *cancel_conn = PQrequestCancelStart(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQstatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGconn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQsocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQfinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQerrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQfinish(cancel_conn);
+ return failed;
}
+ PQfinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 11e9b4e8cc..2608f63d79 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2567,6 +2567,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 6b5de89e14..e17a9569b4 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -681,6 +682,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 1c20901c3c..45f8001fbd 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8850,10 +8982,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index a30c66f13a..ff18dab043 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -162,19 +162,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index cf554d389f..e8356e75a2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 24a598b6e4..8a2a7c112c 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..50b6a7bc7d 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..ed625d486e 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,230 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(10)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ pg_usleep(10000);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1769,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1867,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.17.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-06-25 00:36 ` Justin Pryzby <[email protected]>
2022-06-27 11:45 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
1 sibling, 1 reply; 34+ messages in thread
From: Justin Pryzby @ 2022-06-25 00:36 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
Resending with a problematic email removed from CC...
On Mon, Apr 04, 2022 at 03:21:54PM +0000, Jelte Fennema wrote:
> 2. Added some extra sleeps to the cancellation test, to remove random failures on FreeBSD.
Apparently there's still an occasional issue.
https://cirrus-ci.com/task/6613309985128448
result 232/352 (error): ERROR: duplicate key value violates unique constraint "ppln_uniqviol_pkey"
DETAIL: Key (id)=(116) already exists.
This shows that the issue is pretty rare:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/38/3511
--
Justin
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-25 00:36 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
@ 2022-06-27 11:45 ` Justin Pryzby <[email protected]>
2022-06-27 12:29 ` Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Justin Pryzby @ 2022-06-27 11:45 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
On Fri, Jun 24, 2022 at 07:36:16PM -0500, Justin Pryzby wrote:
> Resending with a problematic email removed from CC...
>
> On Mon, Apr 04, 2022 at 03:21:54PM +0000, Jelte Fennema wrote:
> > 2. Added some extra sleeps to the cancellation test, to remove random failures on FreeBSD.
>
> Apparently there's still an occasional issue.
> https://cirrus-ci.com/task/6613309985128448
I think that failure is actually not related to this patch.
There are probably others, but I noticed because it also affected one of my
patches, which changes nothing relevant.
https://cirrus-ci.com/task/5904044051922944
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-25 00:36 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 11:45 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
@ 2022-06-27 12:29 ` Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 34+ messages in thread
From: Alvaro Herrera @ 2022-06-27 12:29 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
On 2022-Jun-27, Justin Pryzby wrote:
> On Fri, Jun 24, 2022 at 07:36:16PM -0500, Justin Pryzby wrote:
> > Apparently there's still an occasional issue.
> > https://cirrus-ci.com/task/6613309985128448
>
> I think that failure is actually not related to this patch.
Yeah, it's not -- Kyotaro diagnosed it as a problem in libpq's pipeline
mode. I hope to push his fix soon, but there are nearby problems that I
haven't been able to track down a good fix for. I'm looking into the
whole.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-06-27 09:29 ` Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
1 sibling, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-06-27 09:29 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Tom Lane <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
(resent because it was blocked from the mailing-list due to inclusion of a blocked email address in the To line)
From: Andres Freund <[email protected]>
> On 2022-04-04 15:21:54 +0000, Jelte Fennema wrote:
> > 2. Added some extra sleeps to the cancellation test, to remove random
> > failures on FreeBSD.
>
> That's extremely extremely rarely the solution to address test reliability
> issues. It'll fail when running test under valgrind etc.
>
> Why do you need sleeps / can you find another way to make the test reliable?
The problem they are solving is racy behaviour between sending the query
and sending the cancellation. If the cancellation is handled before the query
is started, then the query doesn't get cancelled. To solve this problem I used
the sleeps to wait a bit before sending the cancelation request.
When I wrote this, I couldn't think of a better way to do it then with sleeps.
But I didn't like it either (and I still don't). These emails made me start to think
again, about other ways of solving the problem. I think I've found another
solution (see attached patch). The way I solve it now is by using another
connection to check the state of the first one.
Jelte
Attachments:
[application/octet-stream] 0001-Add-documentation-for-libpq_pipeline-tests.patch (1.5K, ../../PR3PR83MB04765D30B272D65DE9E42CF1F7B99@PR3PR83MB0476.EURPRD83.prod.outlook.com/2-0001-Add-documentation-for-libpq_pipeline-tests.patch)
download | inline diff:
From 2be20b649243934d4fec1829d56fda7fc05b7268 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Thu, 13 Jan 2022 15:26:35 +0100
Subject: [PATCH 1/2] Add documentation for libpq_pipeline tests
This adds some explanation on how to run and add libpq tests.
---
src/test/modules/libpq_pipeline/README | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
index d8174dd579..6eda6c5756 100644
--- a/src/test/modules/libpq_pipeline/README
+++ b/src/test/modules/libpq_pipeline/README
@@ -1 +1,21 @@
Test programs and libraries for libpq
+=====================================
+
+You can manually run a specific test by running:
+
+ ./libpq_pipeline <name of test>
+
+To add a new libpq test to this module you need to edit libpq_pipeline.c. There
+you should add the name of your new test to the "print_test_list" function.
+Then in main you should do something when this test name is passed to the
+program.
+
+If the order in which postgres protocol messages are sent is deterministic for
+your test. Then you can generate a trace of these messages using the following
+command:
+
+ ./libpq_pipeline mynewtest -t traces/mynewtest.trace
+
+Once you've done that you should make sure that when running "make check"
+the generated trace is compared to the expected trace. This is done by adding
+your test name to the $cmptrace definition in the t/001_libpq_pipeline.pl file
--
2.34.1
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (52.1K, ../../PR3PR83MB04765D30B272D65DE9E42CF1F7B99@PR3PR83MB0476.EURPRD83.prod.outlook.com/3-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 437b098b2ec6334affb2d818cf9154c7f170e2dc Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does four things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe but
blocking cancel (without having a need for signal-safety).
4. Use this new cancellation API everywhere in the codebase where
signal-safety is not a necessity.
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing
documentation for all the newly added functions.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQrequestCancelStart can now be used
instead, to have a non-blocking way of sending cancel requests. The
postgres_fdw cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 28 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 274 +++++++++++++-
15 files changed, 894 insertions(+), 153 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a561d1d652..b572cf6d5b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1379,22 +1379,30 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 061ffaf329..fa47274d15 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGconn *cancel_conn = PQrequestCancelStart(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQstatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGconn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQsocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQfinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQerrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQfinish(cancel_conn);
+ return failed;
}
+ PQfinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..6d108b56ef 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2567,6 +2567,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 92d1212027..7e02ed6803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -681,6 +682,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 37ec3cb4e5..8e033bb8f3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8856,10 +8988,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index f2e583f9fa..b9f0c0558c 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -158,19 +158,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 6e936bbff3..7390fbec7c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -379,6 +379,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -605,8 +606,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 8117cbd40f..c553a74898 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..b9ce1d58c1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..52503907bd 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,275 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ if (PQsendQuery(conn, "SELECT pg_sleep(30)") != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep(30)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1814,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1912,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-09-14 21:53 ` Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Tom Lane @ 2022-09-14 21:53 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Jelte Fennema <[email protected]> writes:
> [ non-blocking PQcancel ]
I pushed the 0001 patch (libpq_pipeline documentation) with a bit
of further wordsmithing.
As for 0002, I'm not sure that's anywhere near ready. I doubt it's
a great idea to un-deprecate PQrequestCancel with a major change
in its behavior. If there is anybody out there still using it,
they're not likely to appreciate that. Let's leave that alone and
pick some other name.
I'm also finding the entire design of PQrequestCancelStart etc to
be horribly confusing --- it's not *bad* necessarily, but the chosen
function names are seriously misleading. PQrequestCancelStart doesn't
actually "start" anything, so the apparent parallel with PQconnectStart
is just wrong. It's also fairly unclear what the state of a cancel
PQconn is after the request cycle is completed, and whether you can
re-use it (especially after a failed request), and whether you have
to dispose of it separately.
On the whole it feels like a mistake to have two separate kinds of
PGconn with fundamentally different behaviors and yet no distinction
in the API. I think I'd recommend having a separate struct type
(which might internally contain little more than a pointer to a
cloned PGconn), and provide only a limited set of operations on it.
Seems like create, start/continue cancel request, destroy, and
fetch error message ought to be enough. I don't see a reason why we
need to support all of libpq's inquiry operations on such objects ---
for instance, if you want to know which host is involved, you could
perfectly well query the parent PGconn. Nor do I want to run around
and add code to every single libpq entry point to make it reject cancel
PGconns if it can't support them, but we'd have to do so if there's
just one struct type.
I'm not seeing the use-case for PQconnectComplete. If you want
a non-blocking cancel request, why would you then use a blocking
operation to complete the request? Seems like it'd be better
to have just a monolithic cancel function for those who don't
need non-blocking.
This change:
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
is an absolute non-starter: it breaks ABI for every libpq client,
even ones that aren't using this facility. Why do we need a new
ConnStatusType value anyway? Seems like PostgresPollingStatusType
covers what we need: once you reach PGRES_POLLING_OK, the cancel
request is done.
The test case is still not very bulletproof on slow machines,
as it seems to be assuming that 30 seconds == forever. It
would be all right to use $PostgreSQL::Test::Utils::timeout_default,
but I'm not sure that that's easily retrievable by C code.
Maybe make the TAP test pass it in with another optional switch
to libpq_pipeline? Alternatively, we could teach libpq_pipeline
to do getenv("PG_TEST_TIMEOUT_DEFAULT") with a fallback to 180,
but that feels like it might be overly familiar with the innards
of Utils.pm.
regards, tom lane
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
@ 2022-10-05 13:23 ` Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-10-05 13:23 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Thanks for all the feedback. I attached a new patch that I think
addresses all of it. Below some additional info.
> On the whole it feels like a mistake to have two separate kinds of
> PGconn with fundamentally different behaviors and yet no distinction
> in the API. I think I'd recommend having a separate struct type
> (which might internally contain little more than a pointer to a
> cloned PGconn), and provide only a limited set of operations on it.
In my first version of this patch, this is exactly what I did. But then
I got this feedback from Jacob, so I changed it to reusing PGconn:
> > /* issue a cancel request */
> > extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
> > +extern PGcancelConn * PQcancelConnectStart(PGconn *conn);
> > +extern PGcancelConn * PQcancelConnect(PGconn *conn);
> > +extern PostgresPollingStatusType PQcancelConnectPoll(PGcancelConn * cancelConn);
> > +extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
> > +extern int PQcancelSocket(const PGcancelConn * cancelConn);
> > +extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
> > +extern void PQcancelFinish(PGcancelConn * cancelConn);
>
> That's a lot of new entry points, most of which don't do anything
> except call their twin after a pointer cast. How painful would it be to
> just use the existing APIs as-is, and error out when calling
> unsupported functions if conn->cancelRequest is true?
I changed it back to use PGcancelConn as per your suggestion and I
agree that the API got better because of it.
> + CONNECTION_CANCEL_FINISHED,
> /* Non-blocking mode only below here */
>
> is an absolute non-starter: it breaks ABI for every libpq client,
> even ones that aren't using this facility.
I removed this now. The main reason was so it was clear that no
queries could be sent over the connection, like is normally the case
when CONNECTION_OK happens. I don't think this is as useful anymore
now that this patch has a dedicated PGcancelStatus function.
NOTE: The CONNECTION_STARTING ConnStatusType is still necessary.
But to keep ABI compatibility I moved it to the end of the enum.
> Alternatively, we could teach libpq_pipeline
> to do getenv("PG_TEST_TIMEOUT_DEFAULT") with a fallback to 180,
> but that feels like it might be overly familiar with the innards
> of Utils.pm.
I went with this approach, because this environment variable was
already used in 2 other places than Utils.pm:
- contrib/test_decoding/sql/twophase.sql
- src/test/isolation/isolationtester.c
So, one more place seemed quite harmless.
P.S. I noticed a logical conflict between this patch and my libpq load
balancing patch. Because this patch depends on the connhost array
is constructed the exact same on the second invocation of connectOptions2.
But the libpq loadbalancing patch breaks this assumption. I'm making
a mental (and public) note that whichever of these patches gets merged last
should address this issue.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (52.0K, ../../DBBPR83MB0507788F756288750A6327F8F7469@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 437b098b2ec6334affb2d818cf9154c7f170e2dc Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH] Add non-blocking version of PQcancel
This patch does four things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe but
blocking cancel (without having a need for signal-safety).
4. Use this new cancellation API everywhere in the codebase where
signal-safety is not a necessity.
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing
documentation for all the newly added functions.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQrequestCancelStart can now be used
instead, to have a non-blocking way of sending cancel requests. The
postgres_fdw cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 28 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 274 +++++++++++++-
15 files changed, 894 insertions(+), 153 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a561d1d652..b572cf6d5b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1379,22 +1379,30 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 061ffaf329..fa47274d15 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGconn *cancel_conn = PQrequestCancelStart(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQstatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGconn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQsocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQfinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQerrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQfinish(cancel_conn);
+ return failed;
}
+ PQfinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..6d108b56ef 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2567,6 +2567,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 92d1212027..7e02ed6803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -681,6 +682,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 37ec3cb4e5..8e033bb8f3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8856,10 +8988,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index f2e583f9fa..b9f0c0558c 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -158,19 +158,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 6e936bbff3..7390fbec7c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -379,6 +379,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -605,8 +606,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 8117cbd40f..c553a74898 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..b9ce1d58c1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..52503907bd 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,275 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ if (PQsendQuery(conn, "SELECT pg_sleep(30)") != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep(30)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1814,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1912,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-11-04 15:58 ` Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jacob Champion @ 2022-11-04 15:58 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 10/5/22 06:23, Jelte Fennema wrote:
> In my first version of this patch, this is exactly what I did. But then
> I got this feedback from Jacob, so I changed it to reusing PGconn:
>
>> [snip]
>
> I changed it back to use PGcancelConn as per your suggestion and I
> agree that the API got better because of it.
Sorry for the whiplash!
Is the latest attachment the correct version? I don't see any difference
between the latest 0001 and the previous version's 0002 -- it has no
references to PG_TEST_TIMEOUT_DEFAULT, PGcancelConn, etc.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
@ 2022-11-15 11:38 ` Jelte Fennema <[email protected]>
2022-11-29 19:17 ` Re: Add non-blocking version of PQcancel Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-11-15 11:38 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Ugh, it indeed seems like I somehow messed up sending the new patch.
Here's the correct one.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (55.8K, ../../DBBPR83MB0507E785FE96166AB8D9940BF7049@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From d8d581a0033e0365faf96a39e8ce75a6ec9ebf7d Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
3. Use these two new cancellation APIs everywhere in the codebase where
signal-safety is not a necessity.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests. The postgres_fdw
cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 22 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 279 +++++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 375 ++++++++++++++++--
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 25 +-
src/interfaces/libpq/libpq-int.h | 9 +
src/test/isolation/isolationtester.c | 29 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++-
15 files changed, 1050 insertions(+), 109 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 9eef417c47..2a55c6759a 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1378,22 +1378,24 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQcancelSend(conn);
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = "OK";
+ }
+ PQcancelFinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 939d114f02..9622441da7 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQcancelFinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQcancelFinish(cancel_conn);
+ return failed;
}
+ PQcancelFinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index cc9e39c4a5..113f3204cc 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2688,6 +2688,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index e48ccd286b..bf977442d6 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -713,6 +714,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 3c9bd3d673..90db021c1d 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -4909,7 +4909,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -5627,13 +5627,220 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+PGcancelConn *PQcancelSend(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function returns a <structname>PGcancelConn</structname>
+ object. By using
+ <xref linkend="libpq-PQcancelStatus"/>
+ it can be checked if there was any error when sending the cancellation
+ request. If <xref linkend="libpq-PQcancelStatus"/>
+ returns for <symbol>CONNECTION_OK</symbol> the request was
+ successfully sent, but if it returns <symbol>CONNECTION_BAD</symbol>
+ an error occured. If an error occured the error message can be retrieved using
+ <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being cancelled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelSend</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQcancelSend"/> that can be used
+ in a non-blocking manner.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>,
+ but it won't instantly start sending a cancel request over this
+ connection like <xref linkend="libpq-PQcancelSend"/>.
+ <xref linkend="libpq-PQcancelStatus"/> should be called on the return
+ value to check if the <structname> PGcancelConn </structname> was
+ created successfully.
+ The <structname>PGcancelConn</structname> object is an opaque structure
+ that is not meant to be accessed directly by the application.
+ This <structname>PGcancelConn</structname> object can be used to cancel
+ the query that's running on the original connection in a thread-safe and
+ non-blocking way.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually cancelled).
+ While a <symbol>CONNECTION_OK</symbol> result for
+ <structname>PGconn</structname> means thatqueries can be sent over the
+ connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5675,14 +5882,30 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of
+ <xref linkend="libpq-PQcancelSend"/>, but one that can be used safely
+ from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead.
+ <xref linkend="libpq-PQcancel"/> can be safely invoked from a signal
+ handler, if the <parameter>errbuf</parameter> is a local variable in the
+ signal handler. The <structname>PGcancel</structname> object is read-only
+ as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
+ also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -5690,21 +5913,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -5716,13 +5940,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -8871,7 +9104,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 1cc97b72f7..0f5e84ad71 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ PQcancelFinish(PQcancelSend(conn));
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f56e8c185c 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,11 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelSend 187
+PQcancelConn 188
+PQcancelPoll 189
+PQcancelStatus 190
+PQcancelSocket 191
+PQcancelErrorMessage 192
+PQcancelReset 193
+PQcancelFinish 194
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..7b59697e64 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -376,6 +376,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -599,8 +600,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -731,6 +741,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return (PGcancelConn *) cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -907,6 +979,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2055,10 +2167,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConn. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2107,6 +2226,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2247,8 +2375,8 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
@@ -2265,6 +2393,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2274,6 +2430,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2298,6 +2455,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2317,6 +2482,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2498,19 +2672,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConn.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2526,7 +2708,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2536,12 +2718,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2564,7 +2752,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2593,7 +2781,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2687,8 +2875,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2727,6 +2916,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2935,6 +3134,30 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancelation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4114,6 +4337,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a Terminate
+ * message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4582,6 +4814,96 @@ cancel_errReturn:
return false;
}
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+PGcancelConn *
+PQcancelSend(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConn(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn *cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4640,6 +4962,7 @@ PQrequestCancel(PGconn *conn)
}
+
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 795500c593..b5b10ec2ba 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -556,8 +556,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -640,7 +643,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -735,7 +738,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -753,13 +756,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index b42a908733..ca378d2ad5 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -253,7 +253,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 3df4a97f2e..18dff253c4 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -199,6 +199,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is returned.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index b7df3224c0..de2e32ca63 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,28 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* issue a cancel request */
+extern PGcancelConn * PQcancelSend(PGconn *conn);
+/* non-blocking version of PQrequestSend */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn *cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..84027bc4ab 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -397,6 +397,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -592,6 +596,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153..3781f7982b 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelSend(conn);
- if (cancel != NULL)
+ if (PQcancelStatus(cancel_conn) == CONNECTION_OK)
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index c609f42258..2674abb539 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelSend(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1638,6 +1896,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1739,7 +1998,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2022-11-29 19:17 ` Daniel Gustafsson <[email protected]>
2022-11-30 09:20 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Daniel Gustafsson @ 2022-11-29 19:17 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
> On 15 Nov 2022, at 12:38, Jelte Fennema <[email protected]> wrote:
> Here's the correct one.<0001-Add-non-blocking-version-of-PQcancel.patch>
This version of the patch no longer applies, a rebased version is needed.
--
Daniel Gustafsson https://vmware.com/
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-29 19:17 ` Re: Add non-blocking version of PQcancel Daniel Gustafsson <[email protected]>
@ 2022-11-30 09:20 ` Jelte Fennema <[email protected]>
2023-01-19 11:10 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema @ 2022-11-30 09:20 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
> This version of the patch no longer applies, a rebased version is needed.
Attached is a patch that applies cleanly again and is also changed
to use the recently introduced libpq_append_conn_error.
I also attached a patch that runs pgindent after the introduction of
libpq_append_conn_error. I noticed that this hadn't happened when
trying to run pgindent on my own changes.
Attachments:
[application/octet-stream] v9-0001-libpq-Run-pgindent-after-a9e9a9f32b3.patch (39.3K, ../../DBBPR83MB05071BB8A8838830CABAFFE1F7129@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-v9-0001-libpq-Run-pgindent-after-a9e9a9f32b3.patch)
download | inline diff:
From 9bf997762786f9d276f211c588918a1cdca598d5 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 30 Nov 2022 10:07:19 +0100
Subject: [PATCH v9 1/2] libpq: Run pgindent after a9e9a9f32b3
It seems that pgindent was not run after the error handling refactor in
commit a9e9a9f32b35edf129c88e8b929ef223f8511f59. This fixes that and
also addresses a few other things pgindent wanted to change in libpq.
---
src/interfaces/libpq/fe-auth-scram.c | 2 +-
src/interfaces/libpq/fe-auth.c | 8 +-
src/interfaces/libpq/fe-connect.c | 124 +++++++++++------------
src/interfaces/libpq/fe-exec.c | 16 +--
src/interfaces/libpq/fe-lobj.c | 42 ++++----
src/interfaces/libpq/fe-misc.c | 10 +-
src/interfaces/libpq/fe-protocol3.c | 2 +-
src/interfaces/libpq/fe-secure-common.c | 6 +-
src/interfaces/libpq/fe-secure-gssapi.c | 12 +--
src/interfaces/libpq/fe-secure-openssl.c | 64 ++++++------
src/interfaces/libpq/fe-secure.c | 8 +-
src/interfaces/libpq/libpq-int.h | 4 +-
12 files changed, 149 insertions(+), 149 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index e71626580a..b8e961795d 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -710,7 +710,7 @@ read_server_final_message(fe_scram_state *state, char *input)
return false;
}
libpq_append_conn_error(conn, "error received from server in SCRAM exchange: %s",
- errmsg);
+ errmsg);
return false;
}
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 4a6c358bb6..3caca455aa 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -73,7 +73,7 @@ pg_GSS_continue(PGconn *conn, int payloadlen)
if (!ginbuf.value)
{
libpq_append_conn_error(conn, "out of memory allocating GSSAPI buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
if (pqGetnchar(ginbuf.value, payloadlen, conn))
@@ -223,7 +223,7 @@ pg_SSPI_continue(PGconn *conn, int payloadlen)
if (!inputbuf)
{
libpq_append_conn_error(conn, "out of memory allocating SSPI buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
if (pqGetnchar(inputbuf, payloadlen, conn))
@@ -623,7 +623,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
if (!challenge)
{
libpq_append_conn_error(conn, "out of memory allocating SASL buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
@@ -1277,7 +1277,7 @@ PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user,
else
{
libpq_append_conn_error(conn, "unrecognized password encryption algorithm \"%s\"",
- algorithm);
+ algorithm);
return NULL;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f88d672c6c..d0c3b21fb9 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -1079,7 +1079,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "could not match %d host names to %d hostaddr values",
- count_comma_separated_elems(conn->pghost), conn->nconnhost);
+ count_comma_separated_elems(conn->pghost), conn->nconnhost);
return false;
}
}
@@ -1159,7 +1159,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "could not match %d port numbers to %d hosts",
- count_comma_separated_elems(conn->pgport), conn->nconnhost);
+ count_comma_separated_elems(conn->pgport), conn->nconnhost);
return false;
}
}
@@ -1248,7 +1248,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "channel_binding", conn->channel_binding);
+ "channel_binding", conn->channel_binding);
return false;
}
}
@@ -1273,7 +1273,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "sslmode", conn->sslmode);
+ "sslmode", conn->sslmode);
return false;
}
@@ -1293,7 +1293,7 @@ connectOptions2(PGconn *conn)
case 'v': /* "verify-ca" or "verify-full" */
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "sslmode value \"%s\" invalid when SSL support is not compiled in",
- conn->sslmode);
+ conn->sslmode);
return false;
}
#endif
@@ -1313,16 +1313,16 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "ssl_min_protocol_version",
- conn->ssl_min_protocol_version);
+ "ssl_min_protocol_version",
+ conn->ssl_min_protocol_version);
return false;
}
if (!sslVerifyProtocolVersion(conn->ssl_max_protocol_version))
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "ssl_max_protocol_version",
- conn->ssl_max_protocol_version);
+ "ssl_max_protocol_version",
+ conn->ssl_max_protocol_version);
return false;
}
@@ -1359,7 +1359,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when GSSAPI support is not compiled in",
- conn->gssencmode);
+ conn->gssencmode);
return false;
}
#endif
@@ -1392,8 +1392,8 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "target_session_attrs",
- conn->target_session_attrs);
+ "target_session_attrs",
+ conn->target_session_attrs);
return false;
}
}
@@ -1609,7 +1609,7 @@ connectNoDelay(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "could not set socket to TCP no delay mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1787,7 +1787,7 @@ parse_int_param(const char *value, int *result, PGconn *conn,
error:
libpq_append_conn_error(conn, "invalid integer value \"%s\" for connection option \"%s\"",
- value, context);
+ value, context);
return false;
}
@@ -1816,9 +1816,9 @@ setKeepalivesIdle(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- PG_TCP_KEEPALIVE_IDLE_STR,
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ PG_TCP_KEEPALIVE_IDLE_STR,
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1850,9 +1850,9 @@ setKeepalivesInterval(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_KEEPINTVL",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_KEEPINTVL",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1885,9 +1885,9 @@ setKeepalivesCount(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_KEEPCNT",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_KEEPCNT",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1949,8 +1949,8 @@ prepKeepalivesWin32(PGconn *conn)
if (!setKeepalivesWin32(conn->sock, idle, interval))
{
libpq_append_conn_error(conn, "%s(%s) failed: error code %d",
- "WSAIoctl", "SIO_KEEPALIVE_VALS",
- WSAGetLastError());
+ "WSAIoctl", "SIO_KEEPALIVE_VALS",
+ WSAGetLastError());
return 0;
}
return 1;
@@ -1983,9 +1983,9 @@ setTCPUserTimeout(PGconn *conn)
char sebuf[256];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_USER_TIMEOUT",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_USER_TIMEOUT",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -2354,7 +2354,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not translate host name \"%s\" to address: %s",
- ch->host, gai_strerror(ret));
+ ch->host, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2366,7 +2366,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not parse network address \"%s\": %s",
- ch->hostaddr, gai_strerror(ret));
+ ch->hostaddr, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2377,8 +2377,8 @@ keep_going: /* We will come back to here until there is
if (strlen(portstr) >= UNIXSOCK_PATH_BUFLEN)
{
libpq_append_conn_error(conn, "Unix-domain socket path \"%s\" is too long (maximum %d bytes)",
- portstr,
- (int) (UNIXSOCK_PATH_BUFLEN - 1));
+ portstr,
+ (int) (UNIXSOCK_PATH_BUFLEN - 1));
goto keep_going;
}
@@ -2391,7 +2391,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not translate Unix-domain socket path \"%s\" to address: %s",
- portstr, gai_strerror(ret));
+ portstr, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2513,7 +2513,7 @@ keep_going: /* We will come back to here until there is
}
emitHostIdentityInfo(conn, host_addr);
libpq_append_conn_error(conn, "could not create socket: %s",
- SOCK_STRERROR(errorno, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(errorno, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2543,7 +2543,7 @@ keep_going: /* We will come back to here until there is
if (!pg_set_noblock(conn->sock))
{
libpq_append_conn_error(conn, "could not set socket to nonblocking mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
conn->try_next_addr = true;
goto keep_going;
}
@@ -2552,7 +2552,7 @@ keep_going: /* We will come back to here until there is
if (fcntl(conn->sock, F_SETFD, FD_CLOEXEC) == -1)
{
libpq_append_conn_error(conn, "could not set socket to close-on-exec mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
conn->try_next_addr = true;
goto keep_going;
}
@@ -2581,9 +2581,9 @@ keep_going: /* We will come back to here until there is
(char *) &on, sizeof(on)) < 0)
{
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "SO_KEEPALIVE",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "SO_KEEPALIVE",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
err = 1;
}
else if (!setKeepalivesIdle(conn)
@@ -2708,7 +2708,7 @@ keep_going: /* We will come back to here until there is
(char *) &optval, &optlen) == -1)
{
libpq_append_conn_error(conn, "could not get socket error status: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
else if (optval != 0)
@@ -2735,7 +2735,7 @@ keep_going: /* We will come back to here until there is
&conn->laddr.salen) < 0)
{
libpq_append_conn_error(conn, "could not get client address from socket: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2775,7 +2775,7 @@ keep_going: /* We will come back to here until there is
libpq_append_conn_error(conn, "requirepeer parameter is not supported on this platform");
else
libpq_append_conn_error(conn, "could not get peer credentials: %s",
- strerror_r(errno, sebuf, sizeof(sebuf)));
+ strerror_r(errno, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2788,7 +2788,7 @@ keep_going: /* We will come back to here until there is
if (strcmp(remote_username, conn->requirepeer) != 0)
{
libpq_append_conn_error(conn, "requirepeer specifies \"%s\", but actual peer user name is \"%s\"",
- conn->requirepeer, remote_username);
+ conn->requirepeer, remote_username);
free(remote_username);
goto error_return;
}
@@ -2829,7 +2829,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send GSSAPI negotiation packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2840,7 +2840,7 @@ keep_going: /* We will come back to here until there is
else if (!conn->gctx && conn->gssencmode[0] == 'r')
{
libpq_append_conn_error(conn,
- "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
+ "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
goto error_return;
}
#endif
@@ -2882,7 +2882,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send SSL negotiation packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
/* Ok, wait for response */
@@ -2911,7 +2911,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, startpacket, packetlen) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send startup packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
free(startpacket);
goto error_return;
}
@@ -3012,7 +3012,7 @@ keep_going: /* We will come back to here until there is
else
{
libpq_append_conn_error(conn, "received invalid response to SSL negotiation: %c",
- SSLok);
+ SSLok);
goto error_return;
}
}
@@ -3123,7 +3123,7 @@ keep_going: /* We will come back to here until there is
else if (gss_ok != 'G')
{
libpq_append_conn_error(conn, "received invalid response to GSSAPI negotiation: %c",
- gss_ok);
+ gss_ok);
goto error_return;
}
}
@@ -3201,7 +3201,7 @@ keep_going: /* We will come back to here until there is
if (!(beresp == 'R' || beresp == 'v' || beresp == 'E'))
{
libpq_append_conn_error(conn, "expected authentication request from server, but received %c",
- beresp);
+ beresp);
goto error_return;
}
@@ -3216,17 +3216,17 @@ keep_going: /* We will come back to here until there is
* Try to validate message length before using it.
* Authentication requests can't be very large, although GSS
* auth requests may not be that small. Same for
- * NegotiateProtocolVersion. Errors can be a
- * little larger, but not huge. If we see a large apparent
- * length in an error, it means we're really talking to a
- * pre-3.0-protocol server; cope. (Before version 14, the
- * server also used the old protocol for errors that happened
- * before processing the startup packet.)
+ * NegotiateProtocolVersion. Errors can be a little larger,
+ * but not huge. If we see a large apparent length in an
+ * error, it means we're really talking to a pre-3.0-protocol
+ * server; cope. (Before version 14, the server also used the
+ * old protocol for errors that happened before processing the
+ * startup packet.)
*/
if ((beresp == 'R' || beresp == 'v') && (msgLength < 8 || msgLength > 2000))
{
libpq_append_conn_error(conn, "expected authentication request from server, but received %c",
- beresp);
+ beresp);
goto error_return;
}
@@ -3705,7 +3705,7 @@ keep_going: /* We will come back to here until there is
/* Append error report to conn->errorMessage. */
libpq_append_conn_error(conn, "\"%s\" failed",
- "SHOW transaction_read_only");
+ "SHOW transaction_read_only");
/* Close connection politely. */
conn->status = CONNECTION_OK;
@@ -3755,7 +3755,7 @@ keep_going: /* We will come back to here until there is
/* Append error report to conn->errorMessage. */
libpq_append_conn_error(conn, "\"%s\" failed",
- "SELECT pg_is_in_recovery()");
+ "SELECT pg_is_in_recovery()");
/* Close connection politely. */
conn->status = CONNECTION_OK;
@@ -3768,8 +3768,8 @@ keep_going: /* We will come back to here until there is
default:
libpq_append_conn_error(conn,
- "invalid connection state %d, probably indicative of memory corruption",
- conn->status);
+ "invalid connection state %d, probably indicative of memory corruption",
+ conn->status);
goto error_return;
}
@@ -7148,7 +7148,7 @@ pgpassfileWarning(PGconn *conn)
if (sqlstate && strcmp(sqlstate, ERRCODE_INVALID_PASSWORD) == 0)
libpq_append_conn_error(conn, "password retrieved from file \"%s\"",
- conn->pgpassfile);
+ conn->pgpassfile);
}
}
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index da229d632a..88600ce883 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -1444,7 +1444,7 @@ PQsendQueryInternal(PGconn *conn, const char *query, bool newQuery)
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "%s not allowed in pipeline mode",
- "PQsendQuery");
+ "PQsendQuery");
return 0;
}
@@ -1512,7 +1512,7 @@ PQsendQueryParams(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -1558,7 +1558,7 @@ PQsendPrepare(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -1652,7 +1652,7 @@ PQsendQueryPrepared(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -2099,10 +2099,9 @@ PQgetResult(PGconn *conn)
/*
* We're about to return the NULL that terminates the round of
- * results from the current query; prepare to send the results
- * of the next query, if any, when we're called next. If there's
- * no next element in the command queue, this gets us in IDLE
- * state.
+ * results from the current query; prepare to send the results of
+ * the next query, if any, when we're called next. If there's no
+ * next element in the command queue, this gets us in IDLE state.
*/
pqPipelineProcessQueue(conn);
res = NULL; /* query is complete */
@@ -3047,6 +3046,7 @@ pqPipelineProcessQueue(PGconn *conn)
return;
case PGASYNC_IDLE:
+
/*
* If we're in IDLE mode and there's some command in the queue,
* get us into PIPELINE_IDLE mode and process normally. Otherwise
diff --git a/src/interfaces/libpq/fe-lobj.c b/src/interfaces/libpq/fe-lobj.c
index bcd228cef1..50282ff423 100644
--- a/src/interfaces/libpq/fe-lobj.c
+++ b/src/interfaces/libpq/fe-lobj.c
@@ -142,7 +142,7 @@ lo_truncate(PGconn *conn, int fd, size_t len)
if (conn->lobjfuncs->fn_lo_truncate == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_truncate");
+ "lo_truncate");
return -1;
}
@@ -205,7 +205,7 @@ lo_truncate64(PGconn *conn, int fd, pg_int64 len)
if (conn->lobjfuncs->fn_lo_truncate64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_truncate64");
+ "lo_truncate64");
return -1;
}
@@ -395,7 +395,7 @@ lo_lseek64(PGconn *conn, int fd, pg_int64 offset, int whence)
if (conn->lobjfuncs->fn_lo_lseek64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_lseek64");
+ "lo_lseek64");
return -1;
}
@@ -485,7 +485,7 @@ lo_create(PGconn *conn, Oid lobjId)
if (conn->lobjfuncs->fn_lo_create == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_create");
+ "lo_create");
return InvalidOid;
}
@@ -558,7 +558,7 @@ lo_tell64(PGconn *conn, int fd)
if (conn->lobjfuncs->fn_lo_tell64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_tell64");
+ "lo_tell64");
return -1;
}
@@ -667,7 +667,7 @@ lo_import_internal(PGconn *conn, const char *filename, Oid oid)
if (fd < 0)
{ /* error */
libpq_append_conn_error(conn, "could not open file \"%s\": %s",
- filename, strerror_r(errno, sebuf, sizeof(sebuf)));
+ filename, strerror_r(errno, sebuf, sizeof(sebuf)));
return InvalidOid;
}
@@ -723,8 +723,8 @@ lo_import_internal(PGconn *conn, const char *filename, Oid oid)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not read from file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return InvalidOid;
}
@@ -778,8 +778,8 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not open file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return -1;
}
@@ -799,8 +799,8 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not write to file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return -1;
}
}
@@ -822,7 +822,7 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
if (close(fd) != 0 && result >= 0)
{
libpq_append_conn_error(conn, "could not write to file \"%s\": %s",
- filename, strerror_r(errno, sebuf, sizeof(sebuf)));
+ filename, strerror_r(errno, sebuf, sizeof(sebuf)));
result = -1;
}
@@ -954,56 +954,56 @@ lo_initialize(PGconn *conn)
if (lobjfuncs->fn_lo_open == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_open");
+ "lo_open");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_close == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_close");
+ "lo_close");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_creat == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_creat");
+ "lo_creat");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_unlink == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_unlink");
+ "lo_unlink");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_lseek == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_lseek");
+ "lo_lseek");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_tell == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_tell");
+ "lo_tell");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_read == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "loread");
+ "loread");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_write == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lowrite");
+ "lowrite");
free(lobjfuncs);
return -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 4159610f6c..fadc46817b 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -749,8 +749,8 @@ retry4:
*/
definitelyEOF:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
@@ -1067,7 +1067,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s() failed: %s", "select",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
}
return result;
@@ -1280,7 +1280,7 @@ libpq_ngettext(const char *msgid, const char *msgid_plural, unsigned long n)
* newline.
*/
void
-libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...)
+libpq_append_error(PQExpBuffer errorMessage, const char *fmt,...)
{
int save_errno = errno;
bool done;
@@ -1309,7 +1309,7 @@ libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...)
* format should not end with a newline.
*/
void
-libpq_append_conn_error(PGconn *conn, const char *fmt, ...)
+libpq_append_conn_error(PGconn *conn, const char *fmt,...)
{
int save_errno = errno;
bool done;
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 364bad2b88..7a8e9c9962 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -466,7 +466,7 @@ static void
handleSyncLoss(PGconn *conn, char id, int msgLength)
{
libpq_append_conn_error(conn, "lost synchronization with server: got message type \"%c\", length %d",
- id, msgLength);
+ id, msgLength);
/* build an error result holding the error message */
pqSaveErrorResult(conn);
conn->asyncStatus = PGASYNC_READY; /* drop out of PQgetResult wait loop */
diff --git a/src/interfaces/libpq/fe-secure-common.c b/src/interfaces/libpq/fe-secure-common.c
index 7e4246c51f..6377e800dd 100644
--- a/src/interfaces/libpq/fe-secure-common.c
+++ b/src/interfaces/libpq/fe-secure-common.c
@@ -226,7 +226,7 @@ pq_verify_peer_name_matches_certificate_ip(PGconn *conn,
* wrong given the subject matter.
*/
libpq_append_conn_error(conn, "certificate contains IP address with invalid length %zu",
- iplen);
+ iplen);
return -1;
}
@@ -235,7 +235,7 @@ pq_verify_peer_name_matches_certificate_ip(PGconn *conn,
if (!addrstr)
{
libpq_append_conn_error(conn, "could not convert certificate's IP address to string: %s",
- strerror_r(errno, sebuf, sizeof(sebuf)));
+ strerror_r(errno, sebuf, sizeof(sebuf)));
return -1;
}
@@ -292,7 +292,7 @@ pq_verify_peer_name_matches_certificate(PGconn *conn)
else if (names_examined == 1)
{
libpq_append_conn_error(conn, "server certificate for \"%s\" does not match host name \"%s\"",
- first_name, host);
+ first_name, host);
}
else
{
diff --git a/src/interfaces/libpq/fe-secure-gssapi.c b/src/interfaces/libpq/fe-secure-gssapi.c
index 0ce92dbf43..af95dfa322 100644
--- a/src/interfaces/libpq/fe-secure-gssapi.c
+++ b/src/interfaces/libpq/fe-secure-gssapi.c
@@ -213,8 +213,8 @@ pg_GSS_write(PGconn *conn, const void *ptr, size_t len)
if (output.length > PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "client tried to send oversize GSSAPI packet (%zu > %zu)",
- (size_t) output.length,
- PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32));
+ (size_t) output.length,
+ PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32));
errno = EIO; /* for lack of a better idea */
goto cleanup;
}
@@ -349,8 +349,8 @@ pg_GSS_read(PGconn *conn, void *ptr, size_t len)
if (input.length > PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "oversize GSSAPI packet sent by the server (%zu > %zu)",
- (size_t) input.length,
- PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
+ (size_t) input.length,
+ PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
errno = EIO; /* for lack of a better idea */
return -1;
}
@@ -588,8 +588,8 @@ pqsecure_open_gss(PGconn *conn)
if (input.length > PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "oversize GSSAPI packet sent by the server (%zu > %zu)",
- (size_t) input.length,
- PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
+ (size_t) input.length,
+ PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
return PGRES_POLLING_FAILED;
}
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index bad85359b6..4eb212d15c 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -213,12 +213,12 @@ rloop:
if (result_errno == EPIPE ||
result_errno == ECONNRESET)
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
else
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
}
else
{
@@ -313,12 +313,12 @@ pgtls_write(PGconn *conn, const void *ptr, size_t len)
result_errno = SOCK_ERRNO;
if (result_errno == EPIPE || result_errno == ECONNRESET)
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
else
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
}
else
{
@@ -410,7 +410,7 @@ pgtls_get_peer_certificate_hash(PGconn *conn, size_t *len)
if (algo_type == NULL)
{
libpq_append_conn_error(conn, "could not find digest for NID %s",
- OBJ_nid2sn(algo_nid));
+ OBJ_nid2sn(algo_nid));
return NULL;
}
break;
@@ -962,7 +962,7 @@ initialize_SSL(PGconn *conn)
if (ssl_min_ver == -1)
{
libpq_append_conn_error(conn, "invalid value \"%s\" for minimum SSL protocol version",
- conn->ssl_min_protocol_version);
+ conn->ssl_min_protocol_version);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -988,7 +988,7 @@ initialize_SSL(PGconn *conn)
if (ssl_max_ver == -1)
{
libpq_append_conn_error(conn, "invalid value \"%s\" for maximum SSL protocol version",
- conn->ssl_max_protocol_version);
+ conn->ssl_max_protocol_version);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1032,7 +1032,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read root certificate file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
SSL_CTX_free(SSL_context);
return -1;
@@ -1084,10 +1084,10 @@ initialize_SSL(PGconn *conn)
*/
if (fnbuf[0] == '\0')
libpq_append_conn_error(conn, "could not get home directory to locate root certificate file\n"
- "Either provide the file or change sslmode to disable server certificate verification.");
+ "Either provide the file or change sslmode to disable server certificate verification.");
else
libpq_append_conn_error(conn, "root certificate file \"%s\" does not exist\n"
- "Either provide the file or change sslmode to disable server certificate verification.", fnbuf);
+ "Either provide the file or change sslmode to disable server certificate verification.", fnbuf);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1117,7 +1117,7 @@ initialize_SSL(PGconn *conn)
if (errno != ENOENT && errno != ENOTDIR)
{
libpq_append_conn_error(conn, "could not open certificate file \"%s\": %s",
- fnbuf, strerror_r(errno, sebuf, sizeof(sebuf)));
+ fnbuf, strerror_r(errno, sebuf, sizeof(sebuf)));
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1135,7 +1135,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read certificate file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
SSL_CTX_free(SSL_context);
return -1;
@@ -1234,7 +1234,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not load SSL engine \"%s\": %s",
- engine_str, err);
+ engine_str, err);
SSLerrfree(err);
free(engine_str);
return -1;
@@ -1245,7 +1245,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not initialize SSL engine \"%s\": %s",
- engine_str, err);
+ engine_str, err);
SSLerrfree(err);
ENGINE_free(conn->engine);
conn->engine = NULL;
@@ -1260,7 +1260,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read private SSL key \"%s\" from engine \"%s\": %s",
- engine_colon, engine_str, err);
+ engine_colon, engine_str, err);
SSLerrfree(err);
ENGINE_finish(conn->engine);
ENGINE_free(conn->engine);
@@ -1273,7 +1273,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not load private SSL key \"%s\" from engine \"%s\": %s",
- engine_colon, engine_str, err);
+ engine_colon, engine_str, err);
SSLerrfree(err);
ENGINE_finish(conn->engine);
ENGINE_free(conn->engine);
@@ -1310,10 +1310,10 @@ initialize_SSL(PGconn *conn)
{
if (errno == ENOENT)
libpq_append_conn_error(conn, "certificate present, but not private key file \"%s\"",
- fnbuf);
+ fnbuf);
else
libpq_append_conn_error(conn, "could not stat private key file \"%s\": %m",
- fnbuf);
+ fnbuf);
return -1;
}
@@ -1321,7 +1321,7 @@ initialize_SSL(PGconn *conn)
if (!S_ISREG(buf.st_mode))
{
libpq_append_conn_error(conn, "private key file \"%s\" is not a regular file",
- fnbuf);
+ fnbuf);
return -1;
}
@@ -1378,7 +1378,7 @@ initialize_SSL(PGconn *conn)
if (SSL_use_PrivateKey_file(conn->ssl, fnbuf, SSL_FILETYPE_ASN1) != 1)
{
libpq_append_conn_error(conn, "could not load private key file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
return -1;
}
@@ -1394,7 +1394,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "certificate does not match private key file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
return -1;
}
@@ -1447,7 +1447,7 @@ open_client_SSL(PGconn *conn)
if (r == -1)
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
else
libpq_append_conn_error(conn, "SSL SYSCALL error: EOF detected");
pgtls_close(conn);
@@ -1489,12 +1489,12 @@ open_client_SSL(PGconn *conn)
case SSL_R_VERSION_TOO_LOW:
#endif
libpq_append_conn_error(conn, "This may indicate that the server does not support any SSL protocol version between %s and %s.",
- conn->ssl_min_protocol_version ?
- conn->ssl_min_protocol_version :
- MIN_OPENSSL_TLS_VERSION,
- conn->ssl_max_protocol_version ?
- conn->ssl_max_protocol_version :
- MAX_OPENSSL_TLS_VERSION);
+ conn->ssl_min_protocol_version ?
+ conn->ssl_min_protocol_version :
+ MIN_OPENSSL_TLS_VERSION,
+ conn->ssl_max_protocol_version ?
+ conn->ssl_max_protocol_version :
+ MAX_OPENSSL_TLS_VERSION);
break;
default:
break;
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index e74d3ccf69..215c9a74ed 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -255,14 +255,14 @@ pqsecure_raw_read(PGconn *conn, void *ptr, size_t len)
case EPIPE:
case ECONNRESET:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
break;
default:
libpq_append_conn_error(conn, "could not receive data from server: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
break;
}
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 512762f999..b4bb6db5a7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -888,8 +888,8 @@ extern char *libpq_ngettext(const char *msgid, const char *msgid_plural, unsigne
*/
#undef _
-extern void libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...) pg_attribute_printf(2, 3);
-extern void libpq_append_conn_error(PGconn *conn, const char *fmt, ...) pg_attribute_printf(2, 3);
+extern void libpq_append_error(PQExpBuffer errorMessage, const char *fmt,...) pg_attribute_printf(2, 3);
+extern void libpq_append_conn_error(PGconn *conn, const char *fmt,...) pg_attribute_printf(2, 3);
/*
* These macros are needed to let error-handling code be portable between
--
2.34.1
[application/octet-stream] v9-0002-Add-non-blocking-version-of-PQcancel.patch (55.7K, ../../DBBPR83MB05071BB8A8838830CABAFFE1F7129@DBBPR83MB0507.EURPRD83.prod.outlook.com/3-v9-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 4103ac630049bd0c9ea25105c2220273901077d7 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH v9 2/2] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
3. Use these two new cancellation APIs everywhere in the codebase where
signal-safety is not a necessity.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests. The postgres_fdw
cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 22 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 279 +++++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 373 ++++++++++++++++--
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 25 +-
src/interfaces/libpq/libpq-int.h | 9 +
src/test/isolation/isolationtester.c | 29 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++-
15 files changed, 1048 insertions(+), 109 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 04095a8f0e..05058dc66b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1378,22 +1378,24 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQcancelSend(conn);
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = "OK";
+ }
+ PQcancelFinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index f0c45b00db..ce5f908bb1 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQcancelFinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQcancelFinish(cancel_conn);
+ return failed;
}
+ PQcancelFinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 2ab3f1efaa..1036ebc336 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2688,6 +2688,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 51560429e0..0923f93803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -713,6 +714,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index f9558dec3b..2e21ea6ee7 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -4909,7 +4909,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -5627,13 +5627,220 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+PGcancelConn *PQcancelSend(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function returns a <structname>PGcancelConn</structname>
+ object. By using
+ <xref linkend="libpq-PQcancelStatus"/>
+ it can be checked if there was any error when sending the cancellation
+ request. If <xref linkend="libpq-PQcancelStatus"/>
+ returns for <symbol>CONNECTION_OK</symbol> the request was
+ successfully sent, but if it returns <symbol>CONNECTION_BAD</symbol>
+ an error occured. If an error occured the error message can be retrieved using
+ <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being cancelled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelSend</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQcancelSend"/> that can be used
+ in a non-blocking manner.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>,
+ but it won't instantly start sending a cancel request over this
+ connection like <xref linkend="libpq-PQcancelSend"/>.
+ <xref linkend="libpq-PQcancelStatus"/> should be called on the return
+ value to check if the <structname> PGcancelConn </structname> was
+ created successfully.
+ The <structname>PGcancelConn</structname> object is an opaque structure
+ that is not meant to be accessed directly by the application.
+ This <structname>PGcancelConn</structname> object can be used to cancel
+ the query that's running on the original connection in a thread-safe and
+ non-blocking way.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually cancelled).
+ While a <symbol>CONNECTION_OK</symbol> result for
+ <structname>PGconn</structname> means thatqueries can be sent over the
+ connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5675,14 +5882,30 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of
+ <xref linkend="libpq-PQcancelSend"/>, but one that can be used safely
+ from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead.
+ <xref linkend="libpq-PQcancel"/> can be safely invoked from a signal
+ handler, if the <parameter>errbuf</parameter> is a local variable in the
+ signal handler. The <structname>PGcancel</structname> object is read-only
+ as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
+ also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -5690,21 +5913,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -5716,13 +5940,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -8871,7 +9104,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 1cc97b72f7..0f5e84ad71 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ PQcancelFinish(PQcancelSend(conn));
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f56e8c185c 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,11 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelSend 187
+PQcancelConn 188
+PQcancelPoll 189
+PQcancelStatus 190
+PQcancelSocket 191
+PQcancelErrorMessage 192
+PQcancelReset 193
+PQcancelFinish 194
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d0c3b21fb9..364fd4f032 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -376,6 +376,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -599,8 +600,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -731,6 +741,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -906,6 +978,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2030,10 +2141,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConn. So leave these settings alone for
+ * cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2082,6 +2200,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2222,8 +2349,8 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
@@ -2240,6 +2367,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2249,6 +2404,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2272,6 +2428,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2291,6 +2455,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2466,19 +2639,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConn.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2494,7 +2675,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2504,12 +2685,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
libpq_append_conn_error(conn, "could not create socket: %s",
@@ -2531,7 +2718,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2558,7 +2745,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2650,8 +2837,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2690,6 +2878,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2891,6 +3089,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancelation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4063,6 +4284,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4531,6 +4761,96 @@ cancel_errReturn:
return false;
}
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+PGcancelConn *
+PQcancelSend(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConn(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4589,6 +4909,7 @@ PQrequestCancel(PGconn *conn)
}
+
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index fadc46817b..40a7b9ab71 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -556,8 +556,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -639,7 +642,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -734,7 +737,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -751,13 +754,17 @@ definitelyEOF:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.");
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 4eb212d15c..1b83946aee 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -248,7 +248,7 @@ rloop:
*/
libpq_append_conn_error(conn, "SSL connection has been closed unexpectedly");
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
libpq_append_conn_error(conn, "unrecognized SSL error code: %d", err);
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 215c9a74ed..2cbfa501e2 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -199,6 +199,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is returned.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index b7df3224c0..697e0ed85d 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,28 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* issue a cancel request */
+extern PGcancelConn * PQcancelSend(PGconn *conn);
+/* non-blocking version of PQrequestSend */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b4bb6db5a7..6e117f8b86 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -397,6 +397,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -592,6 +596,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153..3781f7982b 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelSend(conn);
- if (cancel != NULL)
+ if (PQcancelStatus(cancel_conn) == CONNECTION_OK)
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index a37e4e2500..4018c61a3b 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelSend(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1681,6 +1939,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1782,7 +2041,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-29 19:17 ` Re: Add non-blocking version of PQcancel Daniel Gustafsson <[email protected]>
2022-11-30 09:20 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
@ 2023-01-19 11:10 ` Jelte Fennema <[email protected]>
0 siblings, 0 replies; 34+ messages in thread
From: Jelte Fennema @ 2023-01-19 11:10 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Jacob Champion <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Is there anything that is currently blocking this patch? I'd quite
like it to get into PG16.
Especially since I ran into another use case that I would want to use
this patch for recently: Adding an async cancel function to Python
it's psycopg3 library. This library exposes both a Connection class
and an AsyncConnection class (using python its asyncio feature). But
one downside of the AsyncConnection type is that it doesn't have a
cancel method.
I ran into this while changing the PgBouncer tests to use python. And
the cancellation tests were the only tests that required me to use a
ThreadPoolExecutor instead of simply being able to use async-await
style programming:
https://github.com/pgbouncer/pgbouncer/blob/master/test/test_cancel.py#LL9C17-L9C17
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 01:59 vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: vignesh C @ 2024-01-26 01:59 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Wed, 20 Dec 2023 at 19:17, Jelte Fennema-Nio <[email protected]> wrote:
>
> On Thu, 14 Dec 2023 at 13:57, Jelte Fennema-Nio <[email protected]> wrote:
> > I changed all the places that were not adhering to those spellings.
>
> It seems I forgot a /g on my sed command to do this so it turned out I
> missed one that caused the test to fail to compile... Attached is a
> fixed version.
>
> I also updated the patchset to use the EOF detection provided by
> 0a5c46a7a488f2f4260a90843bb9de6c584c7f4e instead of introducing a new
> way of EOF detection using a -2 return value.
CFBot shows that the patch does not apply anymore as in [1]:
patching file doc/src/sgml/libpq.sgml
...
patching file src/interfaces/libpq/exports.txt
Hunk #1 FAILED at 191.
1 out of 1 hunk FAILED -- saving rejects to file
src/interfaces/libpq/exports.txt.rej
patching file src/interfaces/libpq/fe-connect.c
Please post an updated version for the same.
[1] - http://cfbot.cputube.org/patch_46_3511.log
Regards,
Vignesh
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
@ 2024-01-26 10:44 ` Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 10:44 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 02:59, vignesh C <[email protected]> wrote:
> Please post an updated version for the same.
Done.
Attachments:
[application/x-patch] v25-0002-Add-non-blocking-version-of-PQcancel.patch (43.8K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/2-v25-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 5a94d610a4fe138365e2e88c5cec72eba53ed036 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:04 +0100
Subject: [PATCH v25 2/3] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 451 +++++++++++++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 9 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++-
6 files changed, 987 insertions(+), 51 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..f8e3b5953f0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -394,8 +394,10 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
+static void release_conn_hosts(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -623,8 +625,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -755,6 +766,113 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -930,6 +1048,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2361,10 +2518,18 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2506,7 +2671,10 @@ connectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2631,13 +2799,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3279,6 +3451,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4028,8 +4223,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4397,19 +4598,8 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ release_conn_addrinfo(conn);
+ release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4528,6 +4718,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+static void
+release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
@@ -4535,6 +4750,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4588,7 +4812,13 @@ closePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
@@ -5003,6 +5233,179 @@ cancel_errReturn:
return false;
}
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!connectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return connectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using connectDBstart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..ea99ff4efa5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/x-patch] v25-0003-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/3-v25-0003-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From 2ee4f68919d60bd94f7ecfc23a662a22efaf0e62 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v25 3/3] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index d83f6ae8cbc..df0b88e70c7 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 90c8fa4b705..115c3c117a1 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/x-patch] v25-0001-Fix-spelling-of-canceled-cancellation.patch (3.1K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/4-v25-0001-Fix-spelling-of-canceled-cancellation.patch)
download | inline diff:
From 2c43b57abe766a0b817ee85597a326f3ece5ed41 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:31:18 +0100
Subject: [PATCH v25 1/3] Fix spelling of canceled/cancellation
This fixes places where words derived from cancel were not using their
common en-US spelling.
---
doc/src/sgml/event-trigger.sgml | 2 +-
doc/src/sgml/libpq.sgml | 2 +-
src/backend/storage/lmgr/proc.c | 2 +-
src/test/recovery/t/001_stream_rep.pl | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/event-trigger.sgml b/doc/src/sgml/event-trigger.sgml
index 234b4ffd024..a76bd844257 100644
--- a/doc/src/sgml/event-trigger.sgml
+++ b/doc/src/sgml/event-trigger.sgml
@@ -50,7 +50,7 @@
writing anything to the database when running on a standby.
Also, it's recommended to avoid long-running queries in
<literal>login</literal> event triggers. Notes that, for instance,
- cancelling connection in <application>psql</application> wouldn't cancel
+ canceling connection in <application>psql</application> wouldn't cancel
the in-progress <literal>login</literal> trigger.
</para>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 173ab779a08..d0d5aefadc0 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -7625,7 +7625,7 @@ defaultNoticeProcessor(void *arg, const char *message)
is called. It is the ideal time to initialize any
<literal>instanceData</literal> an event procedure may need. Only one
register event will be fired per event handler per connection. If the
- event procedure fails (returns zero), the registration is cancelled.
+ event procedure fails (returns zero), the registration is canceled.
<synopsis>
typedef struct
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4ad96beb87a..e5977548fe2 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -1353,7 +1353,7 @@ ProcSleep(LOCALLOCK *locallock, LockMethod lockMethodTable)
* coding means that there is a tiny chance that the process
* terminates its current transaction and starts a different one
* before we have a change to send the signal; the worst possible
- * consequence is that a for-wraparound vacuum is cancelled. But
+ * consequence is that a for-wraparound vacuum is canceled. But
* that could happen in any case unless we were to do kill() with
* the lock held, which is much more undesirable.
*/
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
index cb988f4d10c..5311ade509b 100644
--- a/src/test/recovery/t/001_stream_rep.pl
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -601,7 +601,7 @@ is( $node_primary->poll_query_until(
ok( pump_until(
$sigchld_bb, $sigchld_bb_timeout,
\$sigchld_bb_stderr, qr/backup is not in progress/),
- 'base backup cleanly cancelled');
+ 'base backup cleanly canceled');
$sigchld_bb->finish();
done_testing();
base-commit: b199eb89c67d737ced55721590f7fc8ff585e837
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
@ 2024-01-26 12:11 ` Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Alvaro Herrera @ 2024-01-26 12:11 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Pushed 0001.
I wonder, would it make sense to put all these new functions in a
separate file fe-cancel.c?
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"World domination is proceeding according to plan" (Andrew Morton)
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
@ 2024-01-26 16:52 ` Jelte Fennema-Nio <[email protected]>
2024-01-26 17:19 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
0 siblings, 2 replies; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 16:52 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 13:11, Alvaro Herrera <[email protected]> wrote:
> I wonder, would it make sense to put all these new functions in a
> separate file fe-cancel.c?
Okay I tried doing that. I think the end result is indeed quite nice,
having all the cancellation related functions together in a file. But
it did require making a bunch of static functions in fe-connect
extern, and adding them to libpq-int.h. On one hand that seems fine to
me, on the other maybe that indicates that this cancellation logic
makes sense to be in the same file as the other connection functions
(in a sense, connecting is all that a cancel request does).
Attachments:
[application/octet-stream] v26-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/2-v26-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From 0a8de21ec8728a474d89d19880d44efae39038ef Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v26 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v26-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/3-v26-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 63fb41b9aa654a6450a4c2f08d8bf204a5916b08 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v26 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 39 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5357b0a9d22..0622fe32253 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66b77e75e18..a0da7356584 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v26-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/4-v26-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From c727e1ccab265c49f7737ba083dd0bf1aa55471e Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v26 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 0622fe32253..8dbc9d2cc57 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4563,15 +4558,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4614,7 +4609,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4628,9 +4623,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4661,9 +4656,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index a0da7356584..b1e1bd6331f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v26-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (26.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/5-v26-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From b3a3e5e659b68f2a9abf1d9af8733ee4888c5c60 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v26 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/fe-cancel.c | 386 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 405 ++----------------------------
src/interfaces/libpq/libpq-int.h | 2 +
src/interfaces/libpq/meson.build | 1 +
4 files changed, 407 insertions(+), 387 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..f1d836d0216
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,386 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!setKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5357b0a9d22 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2269,12 +2267,12 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (!setKeepalivesWin32(conn->sock, idle, interval))
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..66b77e75e18 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,6 +678,8 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: f7cf9494bad3aef1b2ba1cd84376a1e71797ac50
--
2.34.1
[application/octet-stream] v26-0004-Add-non-blocking-version-of-PQcancel.patch (44.1K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/6-v26-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 67f311ca416361668a6865d2d893f8c6d0cb6cd0 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v26 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 304 +++++++++++++++++-
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 ++++++++++++++-
7 files changed, 974 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index f1d836d0216..e9262359453 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -19,6 +19,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
@@ -55,36 +339,36 @@ PQgetCancel(PGconn *conn)
if (conn->pgtcp_user_timeout != NULL)
{
if (!pq_parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
goto fail;
}
if (conn->keepalives != NULL)
{
if (!pq_parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
+ &cancel->keepalives,
+ conn, "keepalives"))
goto fail;
}
if (conn->keepalives_idle != NULL)
{
if (!pq_parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
goto fail;
}
if (conn->keepalives_interval != NULL)
{
if (!pq_parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
goto fail;
}
if (conn->keepalives_count != NULL)
{
if (!pq_parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
goto fail;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8dbc9d2cc57..b63ac63d514 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4541,6 +4634,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4594,7 +4696,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b1e1bd6331f..94990292a04 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
@ 2024-01-26 17:19 ` Alvaro Herrera <[email protected]>
2024-01-26 23:14 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
1 sibling, 1 reply; 34+ messages in thread
From: Alvaro Herrera @ 2024-01-26 17:19 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 2024-Jan-26, Jelte Fennema-Nio wrote:
> Okay I tried doing that. I think the end result is indeed quite nice,
> having all the cancellation related functions together in a file. But
> it did require making a bunch of static functions in fe-connect
> extern, and adding them to libpq-int.h. On one hand that seems fine to
> me, on the other maybe that indicates that this cancellation logic
> makes sense to be in the same file as the other connection functions
> (in a sense, connecting is all that a cancel request does).
Yeah, I see that point of view as well. I like the end result; the
additional protos in libpq-int.h don't bother me. Does anybody else
wants to share their opinion on it? If none, then I'd consider going
ahead with this version.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"We’ve narrowed the problem down to the customer’s pants being in a situation
of vigorous combustion" (Robert Haas, Postgres expert extraordinaire)
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 17:19 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
@ 2024-01-26 23:14 ` Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 23:14 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 18:19, Alvaro Herrera <[email protected]> wrote:
> Yeah, I see that point of view as well. I like the end result; the
> additional protos in libpq-int.h don't bother me. Does anybody else
> wants to share their opinion on it? If none, then I'd consider going
> ahead with this version.
To be clear, I'm +1 on the new file structure (although if people feel
strongly against it, I don't care enough to make a big deal out of
it).
@Alvaro did you have any other comments on the contents of the patch btw?
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
@ 2024-01-28 03:15 ` vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
1 sibling, 1 reply; 34+ messages in thread
From: vignesh C @ 2024-01-28 03:15 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 22:22, Jelte Fennema-Nio <[email protected]> wrote:
>
> On Fri, 26 Jan 2024 at 13:11, Alvaro Herrera <[email protected]> wrote:
> > I wonder, would it make sense to put all these new functions in a
> > separate file fe-cancel.c?
>
>
> Okay I tried doing that. I think the end result is indeed quite nice,
> having all the cancellation related functions together in a file. But
> it did require making a bunch of static functions in fe-connect
> extern, and adding them to libpq-int.h. On one hand that seems fine to
> me, on the other maybe that indicates that this cancellation logic
> makes sense to be in the same file as the other connection functions
> (in a sense, connecting is all that a cancel request does).
CFBot shows that the patch has few compilation errors as in [1]:
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`handle_sigint':
[17:07:07.621] cancel.c:(.text+0x50): undefined reference to `PQcancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`SetCancelConn':
[17:07:07.621] cancel.c:(.text+0x10c): undefined reference to `PQfreeCancel'
[17:07:07.621] /usr/bin/ld: cancel.c:(.text+0x114): undefined
reference to `PQgetCancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`ResetCancelConn':
[17:07:07.621] cancel.c:(.text+0x148): undefined reference to `PQfreeCancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(connect_utils.o): in function
`disconnectDatabase':
[17:07:07.621] connect_utils.c:(.text+0x2fc): undefined reference to
`PQcancelConn'
[17:07:07.621] /usr/bin/ld: connect_utils.c:(.text+0x307): undefined
reference to `PQcancelSend'
[17:07:07.621] /usr/bin/ld: connect_utils.c:(.text+0x30f): undefined
reference to `PQcancelFinish'
[17:07:07.623] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:07.626] collect2: error: ld returned 1 exit status
[17:07:07.626] make[3]: *** [Makefile:31: pg_amcheck] Error 1
[17:07:07.626] make[2]: *** [Makefile:45: all-pg_amcheck-recurse] Error 2
[17:07:07.626] make[2]: *** Waiting for unfinished jobs....
[17:07:08.126] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:08.130] collect2: error: ld returned 1 exit status
[17:07:08.131] make[3]: *** [Makefile:42: initdb] Error 1
[17:07:08.131] make[2]: *** [Makefile:45: all-initdb-recurse] Error 2
[17:07:08.492] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:08.495] collect2: error: ld returned 1 exit status
[17:07:08.496] make[3]: *** [Makefile:50: pg_basebackup] Error 1
[17:07:08.496] make[2]: *** [Makefile:45: all-pg_basebackup-recurse] Error 2
[17:07:09.060] /usr/bin/ld: parallel.o: in function `sigTermHandler':
[17:07:09.060] parallel.c:(.text+0x1aa): undefined reference to `PQcancel'
Please post an updated version for the same.
[1] - https://cirrus-ci.com/task/6210637211107328
Regards,
Vignesh
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
@ 2024-01-28 09:51 ` Jelte Fennema-Nio <[email protected]>
2024-01-28 12:39 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-28 09:51 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Sun, 28 Jan 2024 at 04:15, vignesh C <[email protected]> wrote:
> CFBot shows that the patch has few compilation errors as in [1]:
> [17:07:07.621] /usr/bin/ld:
> ../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
> `handle_sigint':
> [17:07:07.621] cancel.c:(.text+0x50): undefined reference to `PQcancel'
I forgot to update ./configure based builds with the new file, only
meson was working. Also it seems I trimmed the header list fe-cancel.c
a bit too much for OSX, so I added unistd.h back.
Both of those are fixed now.
Attachments:
[application/octet-stream] v27-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/2-v27-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 134d698f01e430fa8f119a4804888e022b33a68e Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v27 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 39 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5357b0a9d22..0622fe32253 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66b77e75e18..a0da7356584 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v27-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/3-v27-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From c9e1b1d66efddf9f15a22daf450757c6d6561775 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v27 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v27-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/4-v27-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From 5851e24c40ee68a6da3837f27a6d0cab18322b94 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v27 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 0622fe32253..8dbc9d2cc57 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4563,15 +4558,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4614,7 +4609,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4628,9 +4623,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4661,9 +4656,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index a0da7356584..b1e1bd6331f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v27-0004-Add-non-blocking-version-of-PQcancel.patch (44.1K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/5-v27-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From badc854b51348b7b36a873bc67886b3d247d6506 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v27 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 304 +++++++++++++++++-
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 ++++++++++++++-
7 files changed, 974 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index 9626f17a9cd..fd2a4f5b209 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
@@ -57,36 +341,36 @@ PQgetCancel(PGconn *conn)
if (conn->pgtcp_user_timeout != NULL)
{
if (!pq_parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
goto fail;
}
if (conn->keepalives != NULL)
{
if (!pq_parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
+ &cancel->keepalives,
+ conn, "keepalives"))
goto fail;
}
if (conn->keepalives_idle != NULL)
{
if (!pq_parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
goto fail;
}
if (conn->keepalives_interval != NULL)
{
if (!pq_parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
goto fail;
}
if (conn->keepalives_count != NULL)
{
if (!pq_parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
goto fail;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8dbc9d2cc57..b63ac63d514 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4541,6 +4634,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4594,7 +4696,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b1e1bd6331f..94990292a04 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/octet-stream] v27-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (26.9K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/6-v27-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From 98d8e783ebf4afa824bec70fd3ed266e2dba6908 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v27 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/fe-cancel.c | 388 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 405 ++----------------------------
src/interfaces/libpq/libpq-int.h | 2 +
src/interfaces/libpq/meson.build | 1 +
5 files changed, 410 insertions(+), 387 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fce17bc72a0..bfcc7cdde99 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -30,6 +30,7 @@ endif
OBJS = \
$(WIN32RES) \
fe-auth-scram.o \
+ fe-cancel.o \
fe-connect.o \
fe-exec.o \
fe-lobj.o \
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..9626f17a9cd
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,388 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <unistd.h>
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!setKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5357b0a9d22 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2269,12 +2267,12 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (!setKeepalivesWin32(conn->sock, idle, interval))
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..66b77e75e18 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,6 +678,8 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: a3a836fb5e51183eae624d43225279306c2285b8
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
@ 2024-01-28 12:39 ` Jelte Fennema-Nio <[email protected]>
2024-01-29 11:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-28 12:39 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Sun, 28 Jan 2024 at 10:51, Jelte Fennema-Nio <[email protected]> wrote:
> Both of those are fixed now.
Okay, there turned out to also be an issue on Windows with
setKeepalivesWin32 not being available in fe-cancel.c. That's fixed
now too (as well as some minor formatting issues).
Attachments:
[application/x-patch] v28-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/2-v28-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 4efbb0c75341f4612f0c5b8d5d3fe3f8f9c3b43c Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v28 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 38 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5d08b4904d3..bc1f6521650 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,30 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 48c10b474f5..4cbad2c2c83 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/x-patch] v28-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/3-v28-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From f1168ac4c3dd758a77be3ceb8c40bacb9aebef8c Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v28 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc1f6521650..aeb3adc0e31 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4562,15 +4557,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4613,7 +4608,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4627,9 +4622,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4660,9 +4655,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4cbad2c2c83..c1ff12dd396 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/x-patch] v28-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (27.7K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/4-v28-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From d5cdd1451ecd9160d285bdfe3cdcf6df452c5249 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v28 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/fe-cancel.c | 387 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 411 ++----------------------------
src/interfaces/libpq/libpq-int.h | 6 +
src/interfaces/libpq/meson.build | 1 +
5 files changed, 416 insertions(+), 390 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fce17bc72a0..bfcc7cdde99 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -30,6 +30,7 @@ endif
OBJS = \
$(WIN32RES) \
fe-auth-scram.o \
+ fe-cancel.o \
fe-connect.o \
fe-exec.o \
fe-lobj.o \
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..ce28d39f3f5
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,387 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <unistd.h>
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!pqSetKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5d08b4904d3 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2233,8 +2231,8 @@ setKeepalivesCount(PGconn *conn)
*
* CAUTION: This needs to be signal safe, since it's used by PQcancel.
*/
-static int
-setKeepalivesWin32(pgsocket sock, int idle, int interval)
+int
+pqSetKeepalivesWin32(pgsocket sock, int idle, int interval)
{
struct tcp_keepalive ka;
DWORD retsize;
@@ -2269,15 +2267,15 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
- if (!setKeepalivesWin32(conn->sock, idle, interval))
+ if (!pqSetKeepalivesWin32(conn->sock, idle, interval))
{
libpq_append_conn_error(conn, "%s(%s) failed: error code %d",
"WSAIoctl", "SIO_KEEPALIVE_VALS",
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..48c10b474f5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,12 +678,18 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
#define pglock_thread() pg_g_threadlock(true)
#define pgunlock_thread() pg_g_threadlock(false)
+#if defined(WIN32) && defined(SIO_KEEPALIVE_VALS)
+extern int pqSetKeepalivesWin32(pgsocket sock, int idle, int interval);
+#endif
+
/* === in fe-exec.c === */
extern void pqSetResultError(PGresult *res, PQExpBuffer errorMessage, int offset);
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: a3a836fb5e51183eae624d43225279306c2285b8
--
2.34.1
[application/x-patch] v28-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/5-v28-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From cb5b87e6e0c9127352013453bc2e944696d0925a Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v28 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/x-patch] v28-0004-Add-non-blocking-version-of-PQcancel.patch (42.8K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/6-v28-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 819ecc80382b93ffa0a9757119c396e4bf667908 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v28 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 +++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 284 ++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++++++-
7 files changed, 964 insertions(+), 38 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index ce28d39f3f5..e37ee0c45ec 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index aeb3adc0e31..2cd95767fdb 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4540,6 +4633,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4593,7 +4695,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c1ff12dd396..e780b62b2bd 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 12:39 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
@ 2024-01-29 11:44 ` Alvaro Herrera <[email protected]>
2024-01-29 12:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 34+ messages in thread
From: Alvaro Herrera @ 2024-01-29 11:44 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 2024-Jan-28, Jelte Fennema-Nio wrote:
> On Sun, 28 Jan 2024 at 10:51, Jelte Fennema-Nio <[email protected]> wrote:
> > Both of those are fixed now.
>
> Okay, there turned out to also be an issue on Windows with
> setKeepalivesWin32 not being available in fe-cancel.c. That's fixed
> now too (as well as some minor formatting issues).
Thanks! I committed 0001 now. I also renamed the new
pq_parse_int_param to pqParseIntParam, for consistency with other
routines there. Please rebase the other patches.
Thanks,
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Thou shalt check the array bounds of all strings (indeed, all arrays), for
surely where thou typest "foo" someone someday shall type
"supercalifragilisticexpialidocious" (5th Commandment for C programmers)
^ permalink raw reply [nested|flat] 34+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 12:39 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-29 11:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
@ 2024-01-29 12:28 ` Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 34+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-29 12:28 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Mon, 29 Jan 2024 at 12:44, Alvaro Herrera <[email protected]> wrote:
> Thanks! I committed 0001 now. I also renamed the new
> pq_parse_int_param to pqParseIntParam, for consistency with other
> routines there. Please rebase the other patches.
Awesome! Rebased, and renamed pq_release_conn_hosts to
pqReleaseConnHosts for the same consistency reasons.
Attachments:
[application/octet-stream] v29-0004-Add-non-blocking-version-of-PQcancel.patch (42.8K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/2-v29-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From c24759d9932d6cf5d9e18c30105e1bb520270de9 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v29 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 +++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 284 ++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++++++-
7 files changed, 964 insertions(+), 38 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index 51f8d8a78c4..7416791d9f0 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pqReleaseConnHosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index dd240d42d70..4add35ec5cf 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2308,10 +2356,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2453,7 +2509,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2578,13 +2637,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3226,6 +3289,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -3975,8 +4061,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4344,6 +4436,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pqReleaseConnHosts(conn);
free(conn->client_encoding_initial);
@@ -4494,6 +4587,15 @@ pqReleaseConnHosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4547,7 +4649,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 07732927a5b..be45d6098a5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -681,6 +690,7 @@ extern int pqSetKeepalivesWin32(pgsocket sock, int idle, int interval);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
extern void pqReleaseConnHosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/octet-stream] v29-0002-libpq-Add-pqReleaseConnHosts-function.patch (2.5K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/3-v29-0002-libpq-Add-pqReleaseConnHosts-function.patch)
download | inline diff:
From b9db005e37e3dce8aa05d4f09e03a1d806bd8bcd Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v29 2/5] libpq: Add pqReleaseConnHosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 38 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index c0dea144a00..079abfca9e2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4349,19 +4349,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pqReleaseConnHosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4480,6 +4468,30 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pqReleaseConnHosts
+ * - Free the host list in the PGconn.
+ */
+void
+pqReleaseConnHosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ff8e0dce776..0d06e260262 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -683,6 +683,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pqReleaseConnHosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
base-commit: 6a1ea02c491d16474a6214603dce40b5b122d4d1
--
2.34.1
[application/octet-stream] v29-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/4-v29-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From bb3c4589264f962e0b833450958e49be4fb1f4a8 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v29 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v29-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/5-v29-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From 01991be38133f43dd89328ca74f0653d1bb4ca27 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v29 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 079abfca9e2..dd240d42d70 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2277,14 +2272,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2347,14 +2342,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2704,7 +2699,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4181,7 +4176,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4233,11 +4228,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4330,7 +4325,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4516,15 +4511,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4567,7 +4562,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4581,9 +4576,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4614,9 +4609,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 0d06e260262..07732927a5b 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -684,6 +684,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
extern void pqReleaseConnHosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
^ permalink raw reply [nested|flat] 34+ messages in thread
end of thread, other threads:[~2024-01-29 12:28 UTC | newest]
Thread overview: 34+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-13 10:12 [PATCH 3/9] Optimize allocations in bringetbitmap Tomas Vondra <[email protected]>
2022-03-24 21:41 Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-04-01 16:13 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-04-04 15:21 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-06-25 00:36 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 11:45 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 12:29 ` Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-29 19:17 ` Re: Add non-blocking version of PQcancel Daniel Gustafsson <[email protected]>
2022-11-30 09:20 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2023-01-19 11:10 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 17:19 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 23:14 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 12:39 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-29 11:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-29 12:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox