public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v18 16/18] WIP: Move xid horizon computation for page level index vacuum to primary.
36+ messages / 12 participants
[nested] [flat]
* [PATCH v18 16/18] WIP: Move xid horizon computation for page level index vacuum to primary.
@ 2018-12-19 20:32 Andres Freund <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Andres Freund @ 2018-12-19 20:32 UTC (permalink / raw)
During recovery we do not know which table AM to go to to compute the
xid horizon. To allow for pluggable storage this therefore moves the
computation to the primary.
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/access/hash/hash_xlog.c | 153 +-----------------------
src/backend/access/hash/hashinsert.c | 18 ++-
src/backend/access/heap/heapam.c | 128 ++++++++++++++++++++
src/backend/access/index/genam.c | 36 ++++++
src/backend/access/nbtree/nbtpage.c | 7 ++
src/backend/access/nbtree/nbtxlog.c | 156 +------------------------
src/backend/access/rmgrdesc/hashdesc.c | 5 +-
src/backend/access/rmgrdesc/nbtdesc.c | 3 +-
src/include/access/genam.h | 5 +
src/include/access/hash_xlog.h | 1 +
src/include/access/heapam.h | 4 +
src/include/access/nbtxlog.h | 1 +
12 files changed, 201 insertions(+), 316 deletions(-)
diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c
index c6d87261579..20441e307a8 100644
--- a/src/backend/access/hash/hash_xlog.c
+++ b/src/backend/access/hash/hash_xlog.c
@@ -969,155 +969,6 @@ hash_xlog_update_meta_page(XLogReaderState *record)
UnlockReleaseBuffer(metabuf);
}
-/*
- * Get the latestRemovedXid from the heap pages pointed at by the index
- * tuples being deleted. See also btree_xlog_delete_get_latestRemovedXid,
- * on which this function is based.
- */
-static TransactionId
-hash_xlog_vacuum_get_latestRemovedXid(XLogReaderState *record)
-{
- xl_hash_vacuum_one_page *xlrec;
- OffsetNumber *unused;
- Buffer ibuffer,
- hbuffer;
- Page ipage,
- hpage;
- RelFileNode rnode;
- BlockNumber blkno;
- ItemId iitemid,
- hitemid;
- IndexTuple itup;
- HeapTupleHeader htuphdr;
- BlockNumber hblkno;
- OffsetNumber hoffnum;
- TransactionId latestRemovedXid = InvalidTransactionId;
- int i;
-
- xlrec = (xl_hash_vacuum_one_page *) XLogRecGetData(record);
-
- /*
- * If there's nothing running on the standby we don't need to derive a
- * full latestRemovedXid value, so use a fast path out of here. This
- * returns InvalidTransactionId, and so will conflict with all HS
- * transactions; but since we just worked out that that's zero people,
- * it's OK.
- *
- * XXX There is a race condition here, which is that a new backend might
- * start just after we look. If so, it cannot need to conflict, but this
- * coding will result in throwing a conflict anyway.
- */
- if (CountDBBackends(InvalidOid) == 0)
- return latestRemovedXid;
-
- /*
- * Check if WAL replay has reached a consistent database state. If not, we
- * must PANIC. See the definition of
- * btree_xlog_delete_get_latestRemovedXid for more details.
- */
- if (!reachedConsistency)
- elog(PANIC, "hash_xlog_vacuum_get_latestRemovedXid: cannot operate with inconsistent data");
-
- /*
- * Get index page. If the DB is consistent, this should not fail, nor
- * should any of the heap page fetches below. If one does, we return
- * InvalidTransactionId to cancel all HS transactions. That's probably
- * overkill, but it's safe, and certainly better than panicking here.
- */
- XLogRecGetBlockTag(record, 0, &rnode, NULL, &blkno);
- ibuffer = XLogReadBufferExtended(rnode, MAIN_FORKNUM, blkno, RBM_NORMAL);
-
- if (!BufferIsValid(ibuffer))
- return InvalidTransactionId;
- LockBuffer(ibuffer, HASH_READ);
- ipage = (Page) BufferGetPage(ibuffer);
-
- /*
- * Loop through the deleted index items to obtain the TransactionId from
- * the heap items they point to.
- */
- unused = (OffsetNumber *) ((char *) xlrec + SizeOfHashVacuumOnePage);
-
- for (i = 0; i < xlrec->ntuples; i++)
- {
- /*
- * Identify the index tuple about to be deleted.
- */
- iitemid = PageGetItemId(ipage, unused[i]);
- itup = (IndexTuple) PageGetItem(ipage, iitemid);
-
- /*
- * Locate the heap page that the index tuple points at
- */
- hblkno = ItemPointerGetBlockNumber(&(itup->t_tid));
- hbuffer = XLogReadBufferExtended(xlrec->hnode, MAIN_FORKNUM,
- hblkno, RBM_NORMAL);
-
- if (!BufferIsValid(hbuffer))
- {
- UnlockReleaseBuffer(ibuffer);
- return InvalidTransactionId;
- }
- LockBuffer(hbuffer, HASH_READ);
- hpage = (Page) BufferGetPage(hbuffer);
-
- /*
- * Look up the heap tuple header that the index tuple points at by
- * using the heap node supplied with the xlrec. We can't use
- * heap_fetch, since it uses ReadBuffer rather than XLogReadBuffer.
- * Note that we are not looking at tuple data here, just headers.
- */
- hoffnum = ItemPointerGetOffsetNumber(&(itup->t_tid));
- hitemid = PageGetItemId(hpage, hoffnum);
-
- /*
- * Follow any redirections until we find something useful.
- */
- while (ItemIdIsRedirected(hitemid))
- {
- hoffnum = ItemIdGetRedirect(hitemid);
- hitemid = PageGetItemId(hpage, hoffnum);
- CHECK_FOR_INTERRUPTS();
- }
-
- /*
- * If the heap item has storage, then read the header and use that to
- * set latestRemovedXid.
- *
- * Some LP_DEAD items may not be accessible, so we ignore them.
- */
- if (ItemIdHasStorage(hitemid))
- {
- htuphdr = (HeapTupleHeader) PageGetItem(hpage, hitemid);
- HeapTupleHeaderAdvanceLatestRemovedXid(htuphdr, &latestRemovedXid);
- }
- else if (ItemIdIsDead(hitemid))
- {
- /*
- * Conjecture: if hitemid is dead then it had xids before the xids
- * marked on LP_NORMAL items. So we just ignore this item and move
- * onto the next, for the purposes of calculating
- * latestRemovedxids.
- */
- }
- else
- Assert(!ItemIdIsUsed(hitemid));
-
- UnlockReleaseBuffer(hbuffer);
- }
-
- UnlockReleaseBuffer(ibuffer);
-
- /*
- * If all heap tuples were LP_DEAD then we will be returning
- * InvalidTransactionId here, which avoids conflicts. This matches
- * existing logic which assumes that LP_DEAD tuples must already be older
- * than the latestRemovedXid on the cleanup record that set them as
- * LP_DEAD, hence must already have generated a conflict.
- */
- return latestRemovedXid;
-}
-
/*
* replay delete operation in hash index to remove
* tuples marked as DEAD during index tuple insertion.
@@ -1149,12 +1000,10 @@ hash_xlog_vacuum_one_page(XLogReaderState *record)
*/
if (InHotStandby)
{
- TransactionId latestRemovedXid =
- hash_xlog_vacuum_get_latestRemovedXid(record);
RelFileNode rnode;
XLogRecGetBlockTag(record, 0, &rnode, NULL, NULL);
- ResolveRecoveryConflictWithSnapshot(latestRemovedXid, rnode);
+ ResolveRecoveryConflictWithSnapshot(xldata->latestRemovedXid, rnode);
}
action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &buffer);
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 970733f0cd4..a248bd0743f 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -23,8 +23,8 @@
#include "storage/buf_internals.h"
#include "storage/predicate.h"
-static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
- RelFileNode hnode);
+static void _hash_vacuum_one_page(Relation rel, Relation hrel,
+ Buffer metabuf, Buffer buf);
/*
* _hash_doinsert() -- Handle insertion of a single index tuple.
@@ -137,7 +137,7 @@ restart_insert:
if (IsBufferCleanupOK(buf))
{
- _hash_vacuum_one_page(rel, metabuf, buf, heapRel->rd_node);
+ _hash_vacuum_one_page(rel, heapRel, metabuf, buf);
if (PageGetFreeSpace(page) >= itemsz)
break; /* OK, now we have enough space */
@@ -335,8 +335,7 @@ _hash_pgaddmultitup(Relation rel, Buffer buf, IndexTuple *itups,
*/
static void
-_hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
- RelFileNode hnode)
+_hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
{
OffsetNumber deletable[MaxOffsetNumber];
int ndeletable = 0;
@@ -360,6 +359,12 @@ _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
if (ndeletable > 0)
{
+ TransactionId latestRemovedXid;
+
+ latestRemovedXid =
+ index_compute_xid_horizon_for_tuples(rel, hrel, buf,
+ deletable, ndeletable);
+
/*
* Write-lock the meta page so that we can decrement tuple count.
*/
@@ -393,7 +398,8 @@ _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
xl_hash_vacuum_one_page xlrec;
XLogRecPtr recptr;
- xlrec.hnode = hnode;
+ xlrec.latestRemovedXid = latestRemovedXid;
+ xlrec.hnode = hrel->rd_node;
xlrec.ntuples = ndeletable;
XLogBeginInsert();
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 0321d6bab9a..e9fdb0d4cf5 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -6946,6 +6946,134 @@ HeapTupleHeaderAdvanceLatestRemovedXid(HeapTupleHeader tuple,
/* *latestRemovedXid may still be invalid at end */
}
+/*
+ * Get the latestRemovedXid from the heap pages pointed at by the index
+ * tuples being deleted.
+ *
+ * This puts the work for calculating latestRemovedXid into the recovery path
+ * rather than the primary path.
+ *
+ * It's possible that this generates a fair amount of I/O, since an index
+ * block may have hundreds of tuples being deleted. To amortize that cost to
+ * some degree, this uses prefetching and combines repeat accesses to the same
+ * block.
+ */
+TransactionId
+heap_compute_xid_horizon_for_tuples(Relation rel,
+ ItemPointerData *tids,
+ int nitems)
+{
+ TransactionId latestRemovedXid = InvalidTransactionId;
+ BlockNumber hblkno;
+ Buffer buf = InvalidBuffer;
+ Page hpage;
+
+ /*
+ * Sort to avoid repeated lookups for the same page, and to make it more
+ * likely to access items in an efficient order. In particular this
+ * ensures thaf if there are multiple pointers to the same page, they all
+ * get processed looking up and locking the page just once.
+ */
+ qsort((void *) tids, nitems, sizeof(ItemPointerData),
+ (int (*) (const void *, const void *)) ItemPointerCompare);
+
+ /* prefetch all pages */
+#ifdef USE_PREFETCH
+ hblkno = InvalidBlockNumber;
+ for (int i = 0; i < nitems; i++)
+ {
+ ItemPointer htid = &tids[i];
+
+ if (hblkno == InvalidBlockNumber ||
+ ItemPointerGetBlockNumber(htid) != hblkno)
+ {
+ hblkno = ItemPointerGetBlockNumber(htid);
+
+ PrefetchBuffer(rel, MAIN_FORKNUM, hblkno);
+ }
+ }
+#endif
+
+ /* Iterate over all tids, and check their horizon */
+ hblkno = InvalidBlockNumber;
+ for (int i = 0; i < nitems; i++)
+ {
+ ItemPointer htid = &tids[i];
+ ItemId hitemid;
+ OffsetNumber hoffnum;
+
+ /*
+ * Read heap buffer, but avoid refetching if it's the same block as
+ * required for the last tid.
+ */
+ if (hblkno == InvalidBlockNumber ||
+ ItemPointerGetBlockNumber(htid) != hblkno)
+ {
+ /* release old buffer */
+ if (BufferIsValid(buf))
+ {
+ LockBuffer(buf, BUFFER_LOCK_UNLOCK);
+ ReleaseBuffer(buf);
+ }
+
+ hblkno = ItemPointerGetBlockNumber(htid);
+
+ buf = ReadBuffer(rel, hblkno);
+ hpage = BufferGetPage(buf);
+
+ LockBuffer(buf, BUFFER_LOCK_SHARE);
+ }
+
+ hoffnum = ItemPointerGetOffsetNumber(htid);
+ hitemid = PageGetItemId(hpage, hoffnum);
+
+ /*
+ * Follow any redirections until we find something useful.
+ */
+ while (ItemIdIsRedirected(hitemid))
+ {
+ hoffnum = ItemIdGetRedirect(hitemid);
+ hitemid = PageGetItemId(hpage, hoffnum);
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ /*
+ * If the heap item has storage, then read the header and use that to
+ * set latestRemovedXid.
+ *
+ * Some LP_DEAD items may not be accessible, so we ignore them.
+ */
+ if (ItemIdHasStorage(hitemid))
+ {
+ HeapTupleHeader htuphdr;
+
+ htuphdr = (HeapTupleHeader) PageGetItem(hpage, hitemid);
+
+ HeapTupleHeaderAdvanceLatestRemovedXid(htuphdr, &latestRemovedXid);
+ }
+ else if (ItemIdIsDead(hitemid))
+ {
+ /*
+ * Conjecture: if hitemid is dead then it had xids before the xids
+ * marked on LP_NORMAL items. So we just ignore this item and move
+ * onto the next, for the purposes of calculating
+ * latestRemovedxids.
+ */
+ }
+ else
+ Assert(!ItemIdIsUsed(hitemid));
+
+ }
+
+ if (BufferIsValid(buf))
+ {
+ LockBuffer(buf, BUFFER_LOCK_UNLOCK);
+ ReleaseBuffer(buf);
+ }
+
+ return latestRemovedXid;
+}
+
/*
* Perform XLogInsert to register a heap cleanup info message. These
* messages are sent once per VACUUM and are required because
diff --git a/src/backend/access/index/genam.c b/src/backend/access/index/genam.c
index d34e4ccd3d5..3dbb264e728 100644
--- a/src/backend/access/index/genam.c
+++ b/src/backend/access/index/genam.c
@@ -273,6 +273,42 @@ BuildIndexValueDescription(Relation indexRelation,
return buf.data;
}
+/*
+ * Get the latestRemovedXid from the heap pages pointed at by the index
+ * tuples being deleted.
+ */
+TransactionId
+index_compute_xid_horizon_for_tuples(Relation irel,
+ Relation hrel,
+ Buffer ibuf,
+ OffsetNumber *itemnos,
+ int nitems)
+{
+ ItemPointerData *htids = (ItemPointerData *) palloc(sizeof(ItemPointerData) * nitems);
+ TransactionId latestRemovedXid = InvalidTransactionId;
+ Page ipage = BufferGetPage(ibuf);
+ IndexTuple itup;
+
+ /* identify what the index tuples about to be deleted point to */
+ for (int i = 0; i < nitems; i++)
+ {
+ ItemId iitemid;
+
+ iitemid = PageGetItemId(ipage, itemnos[i]);
+ itup = (IndexTuple) PageGetItem(ipage, iitemid);
+
+ ItemPointerCopy(&itup->t_tid, &htids[i]);
+ }
+
+ /* determine the actual xid horizon */
+ latestRemovedXid =
+ heap_compute_xid_horizon_for_tuples(hrel, htids, nitems);
+
+ pfree(htids);
+
+ return latestRemovedXid;
+}
+
/* ----------------------------------------------------------------
* heap-or-index-scan access to system catalogs
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 9c785bca95e..bb38bb4606e 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -1032,10 +1032,16 @@ _bt_delitems_delete(Relation rel, Buffer buf,
{
Page page = BufferGetPage(buf);
BTPageOpaque opaque;
+ TransactionId latestRemovedXid = InvalidTransactionId;
/* Shouldn't be called unless there's something to do */
Assert(nitems > 0);
+ if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+ latestRemovedXid =
+ index_compute_xid_horizon_for_tuples(rel, heapRel, buf,
+ itemnos, nitems);
+
/* No ereport(ERROR) until changes are logged */
START_CRIT_SECTION();
@@ -1065,6 +1071,7 @@ _bt_delitems_delete(Relation rel, Buffer buf,
XLogRecPtr recptr;
xl_btree_delete xlrec_delete;
+ xlrec_delete.latestRemovedXid = latestRemovedXid;
xlrec_delete.hnode = heapRel->rd_node;
xlrec_delete.nitems = nitems;
diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c
index b0666b42df3..9c277f5016b 100644
--- a/src/backend/access/nbtree/nbtxlog.c
+++ b/src/backend/access/nbtree/nbtxlog.c
@@ -518,159 +518,6 @@ btree_xlog_vacuum(XLogReaderState *record)
UnlockReleaseBuffer(buffer);
}
-/*
- * Get the latestRemovedXid from the heap pages pointed at by the index
- * tuples being deleted. This puts the work for calculating latestRemovedXid
- * into the recovery path rather than the primary path.
- *
- * It's possible that this generates a fair amount of I/O, since an index
- * block may have hundreds of tuples being deleted. Repeat accesses to the
- * same heap blocks are common, though are not yet optimised.
- *
- * XXX optimise later with something like XLogPrefetchBuffer()
- */
-static TransactionId
-btree_xlog_delete_get_latestRemovedXid(XLogReaderState *record)
-{
- xl_btree_delete *xlrec = (xl_btree_delete *) XLogRecGetData(record);
- OffsetNumber *unused;
- Buffer ibuffer,
- hbuffer;
- Page ipage,
- hpage;
- RelFileNode rnode;
- BlockNumber blkno;
- ItemId iitemid,
- hitemid;
- IndexTuple itup;
- HeapTupleHeader htuphdr;
- BlockNumber hblkno;
- OffsetNumber hoffnum;
- TransactionId latestRemovedXid = InvalidTransactionId;
- int i;
-
- /*
- * If there's nothing running on the standby we don't need to derive a
- * full latestRemovedXid value, so use a fast path out of here. This
- * returns InvalidTransactionId, and so will conflict with all HS
- * transactions; but since we just worked out that that's zero people,
- * it's OK.
- *
- * XXX There is a race condition here, which is that a new backend might
- * start just after we look. If so, it cannot need to conflict, but this
- * coding will result in throwing a conflict anyway.
- */
- if (CountDBBackends(InvalidOid) == 0)
- return latestRemovedXid;
-
- /*
- * In what follows, we have to examine the previous state of the index
- * page, as well as the heap page(s) it points to. This is only valid if
- * WAL replay has reached a consistent database state; which means that
- * the preceding check is not just an optimization, but is *necessary*. We
- * won't have let in any user sessions before we reach consistency.
- */
- if (!reachedConsistency)
- elog(PANIC, "btree_xlog_delete_get_latestRemovedXid: cannot operate with inconsistent data");
-
- /*
- * Get index page. If the DB is consistent, this should not fail, nor
- * should any of the heap page fetches below. If one does, we return
- * InvalidTransactionId to cancel all HS transactions. That's probably
- * overkill, but it's safe, and certainly better than panicking here.
- */
- XLogRecGetBlockTag(record, 0, &rnode, NULL, &blkno);
- ibuffer = XLogReadBufferExtended(rnode, MAIN_FORKNUM, blkno, RBM_NORMAL);
- if (!BufferIsValid(ibuffer))
- return InvalidTransactionId;
- LockBuffer(ibuffer, BT_READ);
- ipage = (Page) BufferGetPage(ibuffer);
-
- /*
- * Loop through the deleted index items to obtain the TransactionId from
- * the heap items they point to.
- */
- unused = (OffsetNumber *) ((char *) xlrec + SizeOfBtreeDelete);
-
- for (i = 0; i < xlrec->nitems; i++)
- {
- /*
- * Identify the index tuple about to be deleted
- */
- iitemid = PageGetItemId(ipage, unused[i]);
- itup = (IndexTuple) PageGetItem(ipage, iitemid);
-
- /*
- * Locate the heap page that the index tuple points at
- */
- hblkno = ItemPointerGetBlockNumber(&(itup->t_tid));
- hbuffer = XLogReadBufferExtended(xlrec->hnode, MAIN_FORKNUM, hblkno, RBM_NORMAL);
- if (!BufferIsValid(hbuffer))
- {
- UnlockReleaseBuffer(ibuffer);
- return InvalidTransactionId;
- }
- LockBuffer(hbuffer, BT_READ);
- hpage = (Page) BufferGetPage(hbuffer);
-
- /*
- * Look up the heap tuple header that the index tuple points at by
- * using the heap node supplied with the xlrec. We can't use
- * heap_fetch, since it uses ReadBuffer rather than XLogReadBuffer.
- * Note that we are not looking at tuple data here, just headers.
- */
- hoffnum = ItemPointerGetOffsetNumber(&(itup->t_tid));
- hitemid = PageGetItemId(hpage, hoffnum);
-
- /*
- * Follow any redirections until we find something useful.
- */
- while (ItemIdIsRedirected(hitemid))
- {
- hoffnum = ItemIdGetRedirect(hitemid);
- hitemid = PageGetItemId(hpage, hoffnum);
- CHECK_FOR_INTERRUPTS();
- }
-
- /*
- * If the heap item has storage, then read the header and use that to
- * set latestRemovedXid.
- *
- * Some LP_DEAD items may not be accessible, so we ignore them.
- */
- if (ItemIdHasStorage(hitemid))
- {
- htuphdr = (HeapTupleHeader) PageGetItem(hpage, hitemid);
-
- HeapTupleHeaderAdvanceLatestRemovedXid(htuphdr, &latestRemovedXid);
- }
- else if (ItemIdIsDead(hitemid))
- {
- /*
- * Conjecture: if hitemid is dead then it had xids before the xids
- * marked on LP_NORMAL items. So we just ignore this item and move
- * onto the next, for the purposes of calculating
- * latestRemovedxids.
- */
- }
- else
- Assert(!ItemIdIsUsed(hitemid));
-
- UnlockReleaseBuffer(hbuffer);
- }
-
- UnlockReleaseBuffer(ibuffer);
-
- /*
- * If all heap tuples were LP_DEAD then we will be returning
- * InvalidTransactionId here, which avoids conflicts. This matches
- * existing logic which assumes that LP_DEAD tuples must already be older
- * than the latestRemovedXid on the cleanup record that set them as
- * LP_DEAD, hence must already have generated a conflict.
- */
- return latestRemovedXid;
-}
-
static void
btree_xlog_delete(XLogReaderState *record)
{
@@ -693,12 +540,11 @@ btree_xlog_delete(XLogReaderState *record)
*/
if (InHotStandby)
{
- TransactionId latestRemovedXid = btree_xlog_delete_get_latestRemovedXid(record);
RelFileNode rnode;
XLogRecGetBlockTag(record, 0, &rnode, NULL, NULL);
- ResolveRecoveryConflictWithSnapshot(latestRemovedXid, rnode);
+ ResolveRecoveryConflictWithSnapshot(xlrec->latestRemovedXid, rnode);
}
/*
diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c
index ade1c618161..a29aa96e9ca 100644
--- a/src/backend/access/rmgrdesc/hashdesc.c
+++ b/src/backend/access/rmgrdesc/hashdesc.c
@@ -113,8 +113,9 @@ hash_desc(StringInfo buf, XLogReaderState *record)
{
xl_hash_vacuum_one_page *xlrec = (xl_hash_vacuum_one_page *) rec;
- appendStringInfo(buf, "ntuples %d",
- xlrec->ntuples);
+ appendStringInfo(buf, "ntuples %d, latest removed xid %u",
+ xlrec->ntuples,
+ xlrec->latestRemovedXid);
break;
}
}
diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c
index 8d5c6ae0ab0..64cf7ed02e4 100644
--- a/src/backend/access/rmgrdesc/nbtdesc.c
+++ b/src/backend/access/rmgrdesc/nbtdesc.c
@@ -56,7 +56,8 @@ btree_desc(StringInfo buf, XLogReaderState *record)
{
xl_btree_delete *xlrec = (xl_btree_delete *) rec;
- appendStringInfo(buf, "%d items", xlrec->nitems);
+ appendStringInfo(buf, "%d items, latest removed xid %u",
+ xlrec->nitems, xlrec->latestRemovedXid);
break;
}
case XLOG_BTREE_MARK_PAGE_HALFDEAD:
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 1936195c535..780f3255de9 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -187,6 +187,11 @@ extern IndexScanDesc RelationGetIndexScan(Relation indexRelation,
extern void IndexScanEnd(IndexScanDesc scan);
extern char *BuildIndexValueDescription(Relation indexRelation,
Datum *values, bool *isnull);
+extern TransactionId index_compute_xid_horizon_for_tuples(Relation irel,
+ Relation hrel,
+ Buffer ibuf,
+ OffsetNumber *itemnos,
+ int nitems);
/*
* heap-or-index access to system catalogs (in genam.c)
diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h
index 9cef1b7c25d..045e2bf58ba 100644
--- a/src/include/access/hash_xlog.h
+++ b/src/include/access/hash_xlog.h
@@ -263,6 +263,7 @@ typedef struct xl_hash_init_bitmap_page
*/
typedef struct xl_hash_vacuum_one_page
{
+ TransactionId latestRemovedXid;
RelFileNode hnode;
int ntuples;
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index a0cbea9ba00..fefe1daea74 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -181,6 +181,10 @@ extern void simple_heap_update(Relation relation, ItemPointer otid,
extern void heap_sync(Relation relation);
+extern TransactionId heap_compute_xid_horizon_for_tuples(Relation rel,
+ ItemPointerData *items,
+ int nitems);
+
/* in heap/pruneheap.c */
extern void heap_page_prune_opt(Relation relation, Buffer buffer);
extern int heap_page_prune(Relation relation, Buffer buffer,
diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h
index a605851c981..a294bd6fef4 100644
--- a/src/include/access/nbtxlog.h
+++ b/src/include/access/nbtxlog.h
@@ -123,6 +123,7 @@ typedef struct xl_btree_split
*/
typedef struct xl_btree_delete
{
+ TransactionId latestRemovedXid;
RelFileNode hnode; /* RelFileNode of the heap the index currently
* points at */
int nitems;
--
2.21.0.dirty
--yvn3crbc4qf4vymf
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0017-tableam-Add-function-to-determine-newest-xid-amo.patch"
^ permalink raw reply [nested|flat] 36+ messages in thread
* Add non-blocking version of PQcancel
@ 2022-01-12 15:22 Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-01-12 15:22 UTC (permalink / raw)
To: pgsql-hackers
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQcancelConnectionStart. This API can be used to send cancellations in a
non-blocking fashion. To do this it internally uses regular PGconn
connection establishment. This has as a downside that
PQcancelConnectionStart cannot be safely called from a signal handler.
Luckily, this should be fine for most usages of this API. Since most
code that's using an event loop handles signals in that event loop as
well (as opposed to calling functions from the signal handler directly).
There are also a few advantages of this approach:
1. No need to add and maintain a second non-blocking connection
establishment codepath.
2. Cancel connections benefit automatically from any improvements made
to the normal connection establishment codepath. Examples of things
that it currently gets for free currently are TLS support and
keepalive settings.
This patch also includes a test for this new API (and also the already
existing cancellation APIs). The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
NOTE: I have not tested this with GSS for the moment. My expectation is
that using this new API with a GSS connection will result in a
CONNECTION_BAD status when calling PQcancelStatus. The reason for this
is that GSS reads will also need to communicate back that an EOF was
found, just like I've done for TLS reads and unencrypted reads. Since in
case of a cancel connection an EOF is actually expected, and should not
be treated as an error.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (19.0K, ../../AM5PR83MB0178D3B31CA1B6EC4A8ECC42F7529@AM5PR83MB0178.EURPRD83.prod.outlook.com/2-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 0e0d747c60d564991fc375f439649ac6f35f4578 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH] Add non-blocking version of PQcancel
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQcancelConnectionStart. This API can be used to send cancellations in a
non-blocking fashion. To do this it internally uses regular PGconn
connection establishment. This has as a downside that
PQcancelConnectionStart cannot be safely called from a signal handler.
Luckily, this should be fine for most usages of this API. Since most
code that's using an event loop handles signals in that event loop as
well (as opposed to calling functions from the signal handler directly).
There are also a few advantages of this approach:
1. No need to add and maintain a second non-blocking connection
establishment codepath.
2. Cancel connections benefit automatically from any improvements made
to the normal connection establishment codepath. Examples of things
that it currently gets for free currently are TLS support and
keepalive settings.
This patch also includes a test for this new API (and also the already
existing cancellation APIs). The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
NOTE: I have not tested this with GSS for the moment. My expectation is
that using this new API with a GSS connection will result in a
CONNECTION_BAD status when calling PQcancelStatus. The reason for this
is that GSS reads will also need to communicate back that an EOF was
found, just like I've done for TLS reads and unencrypted reads. Since in
case of a cancel connection an EOF is actually expected, and should not
be treated as an error.
---
src/interfaces/libpq/exports.txt | 7 +
src/interfaces/libpq/fe-connect.c | 192 +++++++++++++++++-
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 3 +
src/interfaces/libpq/libpq-fe.h | 13 ++
src/interfaces/libpq/libpq-int.h | 8 +
.../modules/libpq_pipeline/libpq_pipeline.c | 115 ++++++++++-
.../libpq_pipeline/t/001_libpq_pipeline.pl | 2 +-
9 files changed, 348 insertions(+), 9 deletions(-)
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..64364afeaf 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,10 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelConnect 187
+PQcancelConnectStart 188
+PQcancelConnectPoll 189
+PQcancelStatus 189
+PQcancelSocket 190
+PQcancelErrorMessage 191
+PQcancelFinish 192
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5fc16be849..347d32ad5f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,11 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +741,120 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConnectStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy over information needed to cancel
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Connect to the database
+ */
+ if (!connectDBStart(cancelConn))
+ {
+ /* Just in case we failed to set it in connectDBStart */
+ cancelConn->status = CONNECTION_BAD;
+ }
+
+ return (PGcancelConn *) cancelConn;
+}
+
+/*
+ * PQcancelConnect
+ *
+ * Cancel a request on the given connection
+ */
+PGcancelConn *
+PQcancelConnect(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConnectStart(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelConnectPoll
+ *
+ * Poll a cancel connection. For usage details see the PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelConnectPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((PGconn *) cancelConn);
+}
+
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQconnectStartParams
*
@@ -914,6 +1032,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2276,6 +2434,17 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+ if (n == -2 && conn->cancelRequest)
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2950,6 +3119,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 7fcfe08fd2..a95d63ffcd 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 9f735ba437..3cd65fa276 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -252,7 +252,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 0b998e254d..b2c66f47a5 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,9 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of clean connection
+ * closure then it returns -2.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 20eb855abc..39aed5db3e 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -57,6 +57,7 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
@@ -163,6 +164,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -327,6 +333,13 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
+extern PGcancelConn * PQcancelConnectStart(PGconn *conn);
+extern PGcancelConn * PQcancelConnect(PGconn *conn);
+extern PostgresPollingStatusType PQcancelConnectPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
/* backwards compatible version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index fcce13843e..8af3dd0ee7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,8 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest;
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -574,6 +576,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -691,6 +698,7 @@ extern int pqPutInt(int value, size_t bytes, PGconn *conn);
extern int pqPutMsgStart(char msg_type, PGconn *conn);
extern int pqPutMsgEnd(PGconn *conn);
extern int pqReadData(PGconn *conn);
+extern int pqReadDataOrEof(PGconn *conn);
extern int pqFlush(PGconn *conn);
extern int pqWait(int forRead, int forWrite, PGconn *conn);
extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..27188d43bb 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,116 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+static void
+confirm_query_cancelled(PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal("PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal("query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal("query failed with a different error than cancellation: %s", PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ char errorbuf[256];
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQrequestcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ PQrequestCancel(conn);
+ confirm_query_cancelled(conn);
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancel = PQgetCancel(conn);
+ PQcancel(cancel, errorbuf, sizeof(errorbuf));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelConnect */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQcancelConnect(conn);
+ if (PQcancelStatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConnectStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQcancelConnectStart(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelConnectPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1555,6 +1665,7 @@ print_test_list(void)
printf("singlerow\n");
printf("transaction\n");
printf("uniqviol\n");
+ printf("cancel\n");
}
int
@@ -1642,7 +1753,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0c164dcaba..e0773543ae 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -26,7 +26,7 @@ for my $testname (@tests)
my @extraargs = ('-r', $numrows);
my $cmptrace = grep(/^$testname$/,
qw(simple_pipeline nosync multi_pipelines prepared singlerow
- pipeline_abort transaction disallowed_in_pipeline)) > 0;
+ pipeline_abort transaction disallowed_in_pipeline cancel)) > 0;
# For a bunch of tests, generate a libpq trace file too.
my $traceout = "$PostgreSQL::Test::Utils::tmp_check/traces/$testname.trace";
--
2.17.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-01-13 00:44 Andres Freund <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Andres Freund @ 2022-01-13 00:44 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: pgsql-hackers
Hi,
On 2022-01-12 15:22:18 +0000, Jelte Fennema wrote:
> This patch also includes a test for this new API (and also the already
> existing cancellation APIs). The test can be easily run like this:
>
> cd src/test/modules/libpq_pipeline
> make && ./libpq_pipeline cancel
Right now tests fails to build on windows with:
[15:45:10.518] src/interfaces/libpq/libpqdll.def : fatal error LNK1121: duplicate ordinal number '189' [c:\cirrus\libpq.vcxproj]
on fails tests on other platforms. See
https://cirrus-ci.com/build/4791821363576832
> NOTE: I have not tested this with GSS for the moment. My expectation is
> that using this new API with a GSS connection will result in a
> CONNECTION_BAD status when calling PQcancelStatus. The reason for this
> is that GSS reads will also need to communicate back that an EOF was
> found, just like I've done for TLS reads and unencrypted reads. Since in
> case of a cancel connection an EOF is actually expected, and should not
> be treated as an error.
The failures do not seem related to this.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-01-13 14:51 Jelte Fennema <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-01-13 14:51 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: pgsql-hackers
Attached is an updated patch which I believe fixes windows and the other test failures.
At least on my machine make check-world passes now when compiled with --enable-tap-tests
I also included a second patch which adds some basic documentation for the libpq tests.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (18.2K, ../../AM5PR83MB0178CE0CEB690F1C041355B7F7539@AM5PR83MB0178.EURPRD83.prod.outlook.com/3-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 3619ddcfc11d7d0cc39cb6dc12a8561fb895b385 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 1/2] Add non-blocking version of PQcancel
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQcancelConnectionStart. This API can be used to send cancellations in a
non-blocking fashion. To do this it internally uses regular PGconn
connection establishment. This has as a downside that
PQcancelConnectionStart cannot be safely called from a signal handler.
Luckily, this should be fine for most usages of this API. Since most
code that's using an event loop handles signals in that event loop as
well (as opposed to calling functions from the signal handler directly).
There are also a few advantages of this approach:
1. No need to add and maintain a second non-blocking connection
establishment codepath.
2. Cancel connections benefit automatically from any improvements made
to the normal connection establishment codepath. Examples of things
that it currently gets for free currently are TLS support and
keepalive settings.
This patch also includes a test for this new API (and also the already
existing cancellation APIs). The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
NOTE: I have not tested this with GSS for the moment. My expectation is
that using this new API with a GSS connection will result in a
CONNECTION_BAD status when calling PQcancelStatus. The reason for this
is that GSS reads will also need to communicate back that an EOF was
found, just like I've done for TLS reads and unencrypted reads. Since in
case of a cancel connection an EOF is actually expected, and should not
be treated as an error.
---
src/interfaces/libpq/exports.txt | 7 +
src/interfaces/libpq/fe-connect.c | 192 +++++++++++++++++-
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 3 +
src/interfaces/libpq/libpq-fe.h | 13 ++
src/interfaces/libpq/libpq-int.h | 8 +
.../modules/libpq_pipeline/libpq_pipeline.c | 115 ++++++++++-
8 files changed, 347 insertions(+), 8 deletions(-)
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..a06214a78f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,10 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelConnect 187
+PQcancelConnectStart 188
+PQcancelConnectPoll 189
+PQcancelStatus 190
+PQcancelSocket 191
+PQcancelErrorMessage 192
+PQcancelFinish 193
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5fc16be849..347d32ad5f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,11 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +741,120 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConnectStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy over information needed to cancel
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Connect to the database
+ */
+ if (!connectDBStart(cancelConn))
+ {
+ /* Just in case we failed to set it in connectDBStart */
+ cancelConn->status = CONNECTION_BAD;
+ }
+
+ return (PGcancelConn *) cancelConn;
+}
+
+/*
+ * PQcancelConnect
+ *
+ * Cancel a request on the given connection
+ */
+PGcancelConn *
+PQcancelConnect(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConnectStart(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelConnectPoll
+ *
+ * Poll a cancel connection. For usage details see the PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelConnectPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((PGconn *) cancelConn);
+}
+
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQconnectStartParams
*
@@ -914,6 +1032,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2276,6 +2434,17 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+ if (n == -2 && conn->cancelRequest)
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2950,6 +3119,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 7fcfe08fd2..a95d63ffcd 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 9f735ba437..3cd65fa276 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -252,7 +252,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 0b998e254d..b2c66f47a5 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,9 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of clean connection
+ * closure then it returns -2.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 20eb855abc..39aed5db3e 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -57,6 +57,7 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
@@ -163,6 +164,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -327,6 +333,13 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
+extern PGcancelConn * PQcancelConnectStart(PGconn *conn);
+extern PGcancelConn * PQcancelConnect(PGconn *conn);
+extern PostgresPollingStatusType PQcancelConnectPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
/* backwards compatible version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index fcce13843e..8af3dd0ee7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,8 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest;
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -574,6 +576,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -691,6 +698,7 @@ extern int pqPutInt(int value, size_t bytes, PGconn *conn);
extern int pqPutMsgStart(char msg_type, PGconn *conn);
extern int pqPutMsgEnd(PGconn *conn);
extern int pqReadData(PGconn *conn);
+extern int pqReadDataOrEof(PGconn *conn);
extern int pqFlush(PGconn *conn);
extern int pqWait(int forRead, int forWrite, PGconn *conn);
extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..27188d43bb 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,116 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+static void
+confirm_query_cancelled(PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal("PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal("query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal("query failed with a different error than cancellation: %s", PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ char errorbuf[256];
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQrequestcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ PQrequestCancel(conn);
+ confirm_query_cancelled(conn);
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancel = PQgetCancel(conn);
+ PQcancel(cancel, errorbuf, sizeof(errorbuf));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelConnect */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQcancelConnect(conn);
+ if (PQcancelStatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConnectStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQcancelConnectStart(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelConnectPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1555,6 +1665,7 @@ print_test_list(void)
printf("singlerow\n");
printf("transaction\n");
printf("uniqviol\n");
+ printf("cancel\n");
}
int
@@ -1642,7 +1753,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.17.1
[application/octet-stream] 0002-Add-documentation-for-libpq_pipeline-tests.patch (1.5K, ../../AM5PR83MB0178CE0CEB690F1C041355B7F7539@AM5PR83MB0178.EURPRD83.prod.outlook.com/4-0002-Add-documentation-for-libpq_pipeline-tests.patch)
download | inline diff:
From e59f8174d5d006e78e474873ca4ba89a4e5c21d2 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Thu, 13 Jan 2022 15:26:35 +0100
Subject: [PATCH 2/2] Add documentation for libpq_pipeline tests
This adds some explanation on how to run and add libpq tests.
---
src/test/modules/libpq_pipeline/README | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
index d8174dd579..3eac0bf131 100644
--- a/src/test/modules/libpq_pipeline/README
+++ b/src/test/modules/libpq_pipeline/README
@@ -1 +1,21 @@
Test programs and libraries for libpq
+=====================================
+
+You can manually run a specific test by running:
+
+ ./libpq_pipeline <name of test>
+
+You can add new tests to libpq_pipeline.c to adding a new test for libpq you
+should add your new test you should add the name of your new test to the
+"print_test_list" function. Then in main you should do something when this test
+name is passed to the program.
+
+If the order in which postgres protocol messages are sent is deterministic for
+your test. Then you can generate a trace of these messages using the following
+command:
+
+ ./libpq_pipeline mynewtest -t traces/mynewtest.trace
+
+Once you've done that you should make sure that when running "make check"
+the generated trace is compared to the expected trace. This is done by adding
+your test name to the $cmptrace definition in the t/001_libpq_pipeline.pl file
--
2.17.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-09 00:27 Jacob Champion <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jacob Champion @ 2022-03-09 00:27 UTC (permalink / raw)
To: [email protected] <[email protected]>; [email protected] <[email protected]>; +Cc: pgsql-hackers
On Thu, 2022-01-13 at 14:51 +0000, Jelte Fennema wrote:
> Attached is an updated patch which I believe fixes windows and the other test failures.
> At least on my machine make check-world passes now when compiled with --enable-tap-tests
>
> I also included a second patch which adds some basic documentation for the libpq tests.
This is not a full review by any means, but here are my thoughts so
far:
> NOTE: I have not tested this with GSS for the moment. My expectation is
> that using this new API with a GSS connection will result in a
> CONNECTION_BAD status when calling PQcancelStatus. The reason for this
> is that GSS reads will also need to communicate back that an EOF was
> found, just like I've done for TLS reads and unencrypted reads.
For what it's worth, I did a smoke test with a Kerberos environment via
./libpq_pipeline cancel '... gssencmode=require'
and the tests claim to pass.
> 2. Cancel connections benefit automatically from any improvements made
> to the normal connection establishment codepath. Examples of things
> that it currently gets for free currently are TLS support and
> keepalive settings.
This seems like a big change compared to PQcancel(); one that's not
really hinted at elsewhere. Having the async version of an API open up
a completely different code path with new features is pretty surprising
to me.
And does the backend actually handle cancel requests via TLS (or GSS)?
It didn't look that way from a quick scan, but I may have missed
something.
> @@ -1555,6 +1665,7 @@ print_test_list(void)
> printf("singlerow\n");
> printf("transaction\n");
> printf("uniqviol\n");
> + printf("cancel\n");
> }
This should probably go near the top; it looks like the existing list
is alphabetized.
The new cancel tests don't print any feedback. It'd be nice to get the
same sort of output as the other tests.
> /* issue a cancel request */
> extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
> +extern PGcancelConn * PQcancelConnectStart(PGconn *conn);
> +extern PGcancelConn * PQcancelConnect(PGconn *conn);
> +extern PostgresPollingStatusType PQcancelConnectPoll(PGcancelConn * cancelConn);
> +extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
> +extern int PQcancelSocket(const PGcancelConn * cancelConn);
> +extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
> +extern void PQcancelFinish(PGcancelConn * cancelConn);
That's a lot of new entry points, most of which don't do anything
except call their twin after a pointer cast. How painful would it be to
just use the existing APIs as-is, and error out when calling
unsupported functions if conn->cancelRequest is true?
--Jacob
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-24 21:41 Tom Lane <[email protected]>
parent: Jacob Champion <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Tom Lane @ 2022-03-24 21:41 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; +Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected]
Jacob Champion <[email protected]> writes:
> On Thu, 2022-01-13 at 14:51 +0000, Jelte Fennema wrote:
>> 2. Cancel connections benefit automatically from any improvements made
>> to the normal connection establishment codepath. Examples of things
>> that it currently gets for free currently are TLS support and
>> keepalive settings.
> This seems like a big change compared to PQcancel(); one that's not
> really hinted at elsewhere. Having the async version of an API open up
> a completely different code path with new features is pretty surprising
> to me.
Well, the patch lacks any user-facing doco at all, so a-fortiori this
point is not covered. I trust the plan was to write docs later.
I kind of feel that this patch is going in the wrong direction.
I do see the need for a version of PQcancel that can encrypt the
transmitted cancel request (and yes, that should work on the backend
side; see recursion in ProcessStartupPacket). I have not seen
requests for a non-blocking version, and this doesn't surprise me.
I feel that the whole non-blocking aspect of libpq probably belongs
to another era when people didn't trust threads.
So what I'd do is make a version that just takes a PGconn, sends the
cancel request, and returns success or failure; never mind the
non-blocking aspect. One possible long-run advantage of this is that
it might be possible to "sync" the cancel request so that we know,
or at least can find out afterwards, exactly which query got
cancelled; something that's fundamentally impossible if the cancel
function works from a clone data structure that is disconnected
from the current connection state.
(Note that it probably makes sense to make a clone PGconn to pass
to fe-connect.c, internally to this function. I just don't want
to expose that to the app.)
regards, tom lane
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-24 22:49 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Andres Freund @ 2022-03-24 22:49 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jacob Champion <[email protected]>; [email protected] <[email protected]>; [email protected]
Hi,
On 2022-03-24 17:41:53 -0400, Tom Lane wrote:
> I kind of feel that this patch is going in the wrong direction.
> I do see the need for a version of PQcancel that can encrypt the
> transmitted cancel request (and yes, that should work on the backend
> side; see recursion in ProcessStartupPacket). I have not seen
> requests for a non-blocking version, and this doesn't surprise me.
> I feel that the whole non-blocking aspect of libpq probably belongs
> to another era when people didn't trust threads.
That's not a whole lot of fun if you think of cases like postgres_fdw (or
citus as in Jelte's case), which run inside the backend. Even with just a
single postgres_fdw, we don't really want to end up in an uninterruptible
PQcancel() that doesn't even react to pg_terminate_backend().
Even if using threads weren't an issue, I don't really buy the premise - most
networking code has moved *away* from using dedicated threads for each
connection. It just doesn't scale.
Leaving PQcancel aside, we use the non-blocking libpq stuff widely
ourselves. I think walreceiver, isolationtester, pgbench etc would be *much*
harder to get working equally well if there was just blocking calls. If
anything, we're getting to the point where purely blocking functionality
shouldn't be added anymore.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-25 18:34 Robert Haas <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Robert Haas @ 2022-03-25 18:34 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 24, 2022 at 6:49 PM Andres Freund <[email protected]> wrote:
> That's not a whole lot of fun if you think of cases like postgres_fdw (or
> citus as in Jelte's case), which run inside the backend. Even with just a
> single postgres_fdw, we don't really want to end up in an uninterruptible
> PQcancel() that doesn't even react to pg_terminate_backend().
>
> Even if using threads weren't an issue, I don't really buy the premise - most
> networking code has moved *away* from using dedicated threads for each
> connection. It just doesn't scale.
>
> Leaving PQcancel aside, we use the non-blocking libpq stuff widely
> ourselves. I think walreceiver, isolationtester, pgbench etc would be *much*
> harder to get working equally well if there was just blocking calls. If
> anything, we're getting to the point where purely blocking functionality
> shouldn't be added anymore.
+1. I think having a non-blocking version of PQcancel() available is a
great idea, and I've wanted it myself. See commit
ae9bfc5d65123aaa0d1cca9988037489760bdeae.
That said, I don't think that this particular patch is going in the
right direction. I think Jacob's comment upthread is right on point:
"This seems like a big change compared to PQcancel(); one that's not
really hinted at elsewhere. Having the async version of an API open up
a completely different code path with new features is pretty
surprising to me." It seems to me that we want to end up with similar
code paths for PQcancel() and the non-blocking version of cancel. We
could get there in two ways. One way would be to implement the
non-blocking functionality in a manner that matches exactly what
PQcancel() does now. I imagine that the existing code from PQcancel()
would move, with some amount of change, into a new set of non-blocking
APIs. Perhaps PQcancel() would then be rewritten to use those new APIs
instead of hand-rolling the same logic. The other possible approach
would be to first change the blocking version of PQcancel() to use the
regular connection code instead of its own idiosyncratic logic, and
then as a second step, extend it with non-blocking interfaces that use
the regular non-blocking connection code. With either of these
approaches, we end up with the functionality working similarly in the
blocking and non-blocking code paths.
Leaving the question of approach aside, I think it's fairly clear that
this patch cannot be seriously considered for v15. One problem is the
lack of user-facing documentation, but there's a other stuff that just
doesn't look sufficiently well-considered. For example, it updates the
comment for pqsecure_read() to say "Returns -1 in case of failures,
except in the case of clean connection closure then it returns -2."
But that function calls any of three different implementation
functions depending on the situation and the patch only updates one of
them. And it updates that function to return -2 when the is
ECONNRESET, which seems to fly in the face of the comment's idea that
this is the "clean connection closure" case. I think it's probably a
bad sign that this function is tinkering with logic in this sort of
low-level function anyway. pqReadData() is a really general function
that manages to work with non-blocking I/O already, so why does
non-blocking query cancellation need to change its return values, or
whether or not it drops data in certain cases?
I'm also skeptical about the fact that we end up with a whole bunch of
new functions that are just wrappers around existing functions. That's
not a scalable approach. Every function that we have for a PGconn will
eventually need a variant that deals with a PGcancelConn. That seems
kind of pointless, especially considering that a PGcancelConn is
*exactly* a PGconn in disguise. If we decide to pursue the approach of
using the existing infrastructure for PGconn objects to handle query
cancellation, we ought to manipulate them using the same functions we
currently do, with some kind of mode or flag or switch or something
that you can use to turn a regular PGconn into something that cancels
a query. Maybe you create the PGconn and call
PQsprinkleMagicCancelDust() on it, and then you just proceed using the
existing functions, or something like that. Then, not only do the
existing functions not need query-cancel analogues, but any new
functions we add in the future don't either.
I'll set the target version for this patch to 16. I hope work continues.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-25 18:46 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Tom Lane @ 2022-03-25 18:46 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
Robert Haas <[email protected]> writes:
> That said, I don't think that this particular patch is going in the
> right direction. I think Jacob's comment upthread is right on point:
> "This seems like a big change compared to PQcancel(); one that's not
> really hinted at elsewhere. Having the async version of an API open up
> a completely different code path with new features is pretty
> surprising to me." It seems to me that we want to end up with similar
> code paths for PQcancel() and the non-blocking version of cancel. We
> could get there in two ways. One way would be to implement the
> non-blocking functionality in a manner that matches exactly what
> PQcancel() does now. I imagine that the existing code from PQcancel()
> would move, with some amount of change, into a new set of non-blocking
> APIs. Perhaps PQcancel() would then be rewritten to use those new APIs
> instead of hand-rolling the same logic. The other possible approach
> would be to first change the blocking version of PQcancel() to use the
> regular connection code instead of its own idiosyncratic logic, and
> then as a second step, extend it with non-blocking interfaces that use
> the regular non-blocking connection code. With either of these
> approaches, we end up with the functionality working similarly in the
> blocking and non-blocking code paths.
I think you misunderstand where the real pain point is. The reason
that PQcancel's functionality is so limited has little to do with
blocking vs non-blocking, and everything to do with the fact that
it's designed to be safe to call from a SIGINT handler. That makes
it quite impractical to invoke OpenSSL, and probably our GSS code
as well. If we want support for all connection-time options then
we have to make a new function that does not promise signal safety.
I'm prepared to yield on the question of whether we should provide
a non-blocking version, though I still say that (a) an easier-to-call,
one-step blocking alternative would be good too, and (b) it should
not be designed around the assumption that there's a completely
independent state object being used to perform the cancel. Even in
the non-blocking case, callers should only deal with the original
PGconn.
> Leaving the question of approach aside, I think it's fairly clear that
> this patch cannot be seriously considered for v15.
Yeah, I don't think it's anywhere near fully baked yet. On the other
hand, we do have a couple of weeks left.
regards, tom lane
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-25 19:22 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Robert Haas @ 2022-03-25 19:22 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 25, 2022 at 2:47 PM Tom Lane <[email protected]> wrote:
> I think you misunderstand where the real pain point is. The reason
> that PQcancel's functionality is so limited has little to do with
> blocking vs non-blocking, and everything to do with the fact that
> it's designed to be safe to call from a SIGINT handler. That makes
> it quite impractical to invoke OpenSSL, and probably our GSS code
> as well. If we want support for all connection-time options then
> we have to make a new function that does not promise signal safety.
Well, that's a fair point, but it's somewhat orthogonal to the one I'm
making, which is that a non-blocking version of function X might be
expected to share code or at least functionality with X itself. Having
something that is named in a way that implies asynchrony without other
differences but which is actually different in other important ways is
no good.
> I'm prepared to yield on the question of whether we should provide
> a non-blocking version, though I still say that (a) an easier-to-call,
> one-step blocking alternative would be good too, and (b) it should
> not be designed around the assumption that there's a completely
> independent state object being used to perform the cancel. Even in
> the non-blocking case, callers should only deal with the original
> PGconn.
Well, this sounds like you're arguing for the first of the two
approaches I thought would be acceptable, rather than the second.
> > Leaving the question of approach aside, I think it's fairly clear that
> > this patch cannot be seriously considered for v15.
>
> Yeah, I don't think it's anywhere near fully baked yet. On the other
> hand, we do have a couple of weeks left.
We do?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-25 19:34 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Tom Lane @ 2022-03-25 19:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected] <[email protected]>; PostgreSQL Hackers <[email protected]>
Robert Haas <[email protected]> writes:
> Well, that's a fair point, but it's somewhat orthogonal to the one I'm
> making, which is that a non-blocking version of function X might be
> expected to share code or at least functionality with X itself. Having
> something that is named in a way that implies asynchrony without other
> differences but which is actually different in other important ways is
> no good.
Yeah. We need to choose a name for these new function(s) that is
sufficiently different from "PQcancel" that people won't expect them
to behave exactly the same as that does. I lack any good ideas about
that, how about you?
>> Yeah, I don't think it's anywhere near fully baked yet. On the other
>> hand, we do have a couple of weeks left.
> We do?
Um, you did read the psql-release discussion about setting the feature
freeze deadline, no?
regards, tom lane
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2022-03-28 09:28 Jelte Fennema <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-03-28 09:28 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; PostgreSQL Hackers <[email protected]>
Thanks for all the feedback everyone. I'll try to send a new patch
later this week that includes user facing docs and a simplified API.
For now a few responses:
> Yeah. We need to choose a name for these new function(s) that is
> sufficiently different from "PQcancel" that people won't expect them
> to behave exactly the same as that does. I lack any good ideas about
> that, how about you?
So I guess the names I proposed were not great, since everyone seems to be falling over them.
But I'd like to make my intention clear with the current naming. After this patch there would be
four different APIs for starting a cancelation:
1. PQrequestCancel: deprecated+old, not signal-safe function for requesting query cancellation, only uses a specific set of connection options
2. PQcancel: Cancel queries in a signal safe way, to be signal-safe it only uses a limited set of connection options
3. PQcancelConnect: Cancel queries in a non-signal safe way that uses all connection options
4. PQcancelConnectStart: Cancel queries in a non-signal safe and non-blocking way that uses all connection options
So the idea was that you should not look at PQcancelConnectStart as the non-blocking
version of PQcancel, but as the non-blocking version of PQcancelConnect. I'll try to
think of some different names too, but IMHO these names could be acceptable
when their differences are addressed sufficiently in the documentation.
One other approach to naming that comes to mind now is repurposing PQrequestCancel:
1. PQrequestCancel: Cancel queries in a non-signal safe way that uses all connection options
2. PQrequestCancelStart: Cancel queries in a non-signal safe and non-blocking way that uses all connection options
3. PQcancel: Cancel queries in a signal safe way, to be signal-safe it only uses a limited set of connection options
> I think it's probably a
> bad sign that this function is tinkering with logic in this sort of
> low-level function anyway. pqReadData() is a really general function
> that manages to work with non-blocking I/O already, so why does
> non-blocking query cancellation need to change its return values, or
> whether or not it drops data in certain cases?
The reason for this low level change is that the cancellation part of the
Postgres protocol is following a different, much more simplistic design
than all the other parts. The client does not expect a response message back
from the server after sending the cancellation request. The expectation
is that the server signals completion by closing the connection, i.e. sending EOF.
For all other parts of the protocol, connection termination should be initiated
client side by sending a Terminate message. So the server closing (sending
EOF) is always unexpected and is thus currently considered an error by pqReadData.
But since this is not the case for the cancellation protocol, the result is
changed to -2 in case of EOF to make it possible to distinguish between
an EOF and an actual error.
> And it updates that function to return -2 when the is
> ECONNRESET, which seems to fly in the face of the comment's idea that
> this is the "clean connection closure" case.
The diff sadly does not include the very relevant comment right above these
lines. Pasting the whole case statement here to clear up this confusion:
case SSL_ERROR_ZERO_RETURN:
/*
* Per OpenSSL documentation, this error code is only returned for
* a clean connection closure, so we should not report it as a
* server crash.
*/
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
n = -2;
break;
> For example, it updates the
> comment for pqsecure_read() to say "Returns -1 in case of failures,
> except in the case of clean connection closure then it returns -2."
> But that function calls any of three different implementation
> functions depending on the situation and the patch only updates one of
> them.
That comment is indeed not describing what is happening correctly and I'll
try to make it clearer. The main reason for it being incorrect is coming from
the fact that receiving EOFs is handled in different places based on the
encryption method:
1. Unencrypted TCP: EOF is not returned as an error by pqsecure_read, but detected by pqReadData (see comments related to definitelyEOF)
2. OpenSSL: EOF is returned as an error by pqsecure_read (see copied case statement above)
3. GSS: When writing the patch I was not sure how EOF handling worked here, but given that the tests passed for Jacob on GSS, I'm guessing it works the same as unencrypted TCP.
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-30 16:08 Jelte Fennema <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-03-30 16:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Jacob Champion <[email protected]>; PostgreSQL Hackers <[email protected]>
I attached a new version of this patch. Which does three main things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe, but
blocking cancel (without having a need for signal safety).
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing documentation
for all these functions.
As a API design change from the previous version, PQrequestCancelStart now
returns a regular PGconn for the cancel connection.
@Tom Lane regarding this:
> Even in the non-blocking case, callers should only deal with the original PGconn.
This would by definition result in non-threadsafe code (afaict). So I refrained from doing this.
The blocking version doesn't expose a PGconn at all, but the non-blocking one now returns a new PGconn.
There's two more changes that I at least want to do before considering this patch mergable:
1. Go over all the functions that can be called with a PGconn, but should not be
called with a cancellation PGconn and error out or exit early.
2. Copy over the SockAddr from the original connection and always connect to
the same socket. I believe with the current code the cancellation could end up
at the wrong server if there are multiple hosts listed in the connection string.
And there's a third item that I would like to do as a bonus:
3. Actually use the non-blocking API for the postgres_fdw code to implement a
timeout. Which would allow this comment can be removed:
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
* amount of time that we might block inside PQgetCancel().
*/
So a next version of this patch can be expected somewhere later this week.
But any feedback on the current version would be appreciated. Because
these 3 changes won't change the overall design much.
Jelte
Attachments:
[application/octet-stream] 0001-Add-documentation-for-libpq_pipeline-tests.patch (1.5K, ../../HE1PR8303MB00735DC38F45BD63A3A49347F71F9@HE1PR8303MB0073.EURPRD83.prod.outlook.com/2-0001-Add-documentation-for-libpq_pipeline-tests.patch)
download | inline diff:
From 22a02899d47d46ed05ada2e38e3f9804981b96eb Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Thu, 13 Jan 2022 15:26:35 +0100
Subject: [PATCH 1/2] Add documentation for libpq_pipeline tests
This adds some explanation on how to run and add libpq tests.
---
src/test/modules/libpq_pipeline/README | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
index d8174dd579..6eda6c5756 100644
--- a/src/test/modules/libpq_pipeline/README
+++ b/src/test/modules/libpq_pipeline/README
@@ -1 +1,21 @@
Test programs and libraries for libpq
+=====================================
+
+You can manually run a specific test by running:
+
+ ./libpq_pipeline <name of test>
+
+To add a new libpq test to this module you need to edit libpq_pipeline.c. There
+you should add the name of your new test to the "print_test_list" function.
+Then in main you should do something when this test name is passed to the
+program.
+
+If the order in which postgres protocol messages are sent is deterministic for
+your test. Then you can generate a trace of these messages using the following
+command:
+
+ ./libpq_pipeline mynewtest -t traces/mynewtest.trace
+
+Once you've done that you should make sure that when running "make check"
+the generated trace is compared to the expected trace. This is done by adding
+your test name to the $cmptrace definition in the t/001_libpq_pipeline.pl file
--
2.17.1
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (39.0K, ../../HE1PR8303MB00735DC38F45BD63A3A49347F71F9@HE1PR8303MB0073.EURPRD83.prod.outlook.com/3-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 98a67a65eee6b7ee1275e48e5053ba8ab3055014 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does two things:
1. Change PQrequestCancel to use the regular connection establishement,
to address a few security issues.
2. Add PQrequestCancelStart which is a thread safe and non blocking
version of this new PQrequestCancel implementation.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns.
This patch adds a new cancellation API to libpq which is called
PQrequestCancelStart. This API can be used to send cancellations in a
non-blocking fashion.
This patch also includes a test for this all of libpq cancellation APIs.
The test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 12 +-
contrib/postgres_fdw/connection.c | 11 +-
doc/src/sgml/libpq.sgml | 212 +++++++++++++----
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 225 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 13 +-
src/interfaces/libpq/libpq-int.h | 2 +
src/test/isolation/isolationtester.c | 29 +--
.../modules/libpq_pipeline/libpq_pipeline.c | 214 ++++++++++++++++-
13 files changed, 627 insertions(+), 126 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a06d4bd12d..30cbb22a22 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1380,22 +1380,14 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
-
- if (res == 1)
+ if (PQrequestCancel(conn))
PG_RETURN_TEXT_P(cstring_to_text("OK"));
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(PQerrorMessage(conn)));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..2e182645f7 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1263,8 +1263,6 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
TimestampTz endtime;
bool timed_out;
@@ -1279,19 +1277,14 @@ pgfdw_cancel_query(PGconn *conn)
* Issue cancel request. Unfortunately, there's no good way to limit the
* amount of time that we might block inside PQgetCancel().
*/
- if ((cancel = PQgetCancel(conn)))
- {
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ if (!PQrequestCancel(conn))
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
+ pchomp(PQerrorMessage(conn)))));
return false;
}
- PQfreeCancel(cancel);
- }
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 0b2a8720f0..6b0683b9b0 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8835,10 +8967,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index a30c66f13a..ff18dab043 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -162,19 +162,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index cf554d389f..5462e1305c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -378,6 +378,7 @@ static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -604,8 +605,11 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +741,58 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Copy over information needed to cancel
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +970,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2134,6 +2230,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2379,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2399,17 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+ if (n == -2 && conn->cancelRequest)
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2419,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2758,6 +2877,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3095,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4342,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4464,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4822,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4854,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
+ PGconn *cancelConn = NULL;
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index d3bf57b850..4ffaea63c1 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -252,7 +252,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..42367d4886 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -165,6 +168,10 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -282,6 +289,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +338,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel */
extern int PQrequestCancel(PGconn *conn);
+/* non blocking and thread safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..ff9555e263 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,8 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest;
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..fe1ca168c8 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,17 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
- {
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ if (PQrequestCancel(conn)) {
+ /*
+ * print to stdout not stderr, as this should appear
+ * in the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..95f1d5eb2f 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,215 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+static void
+confirm_query_cancelled(PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal("PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal("query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal("query failed with a different error than cancellation: %s", PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+static void
+test_cancel(PGconn *conn)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /* test PQcancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (PQsendQuery(conn, "SELECT pg_sleep(3)") != 1)
+ pg_fatal("failed to send query: %s", PQerrorMessage(conn));
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1754,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1852,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.17.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-03-31 05:47 Justin Pryzby <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 2 replies; 36+ messages in thread
From: Justin Pryzby @ 2022-03-31 05:47 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Jacob Champion <[email protected]>; [email protected]
Note that the patch is still variously failing in cirrus.
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/37/3511
You may already know that it's possible to trigger the cirrus ci tasks using a
github branch. See src/tools/ci/README.
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-06-25 00:36 Justin Pryzby <[email protected]>
parent: Justin Pryzby <[email protected]>
1 sibling, 1 reply; 36+ messages in thread
From: Justin Pryzby @ 2022-06-25 00:36 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
Resending with a problematic email removed from CC...
On Mon, Apr 04, 2022 at 03:21:54PM +0000, Jelte Fennema wrote:
> 2. Added some extra sleeps to the cancellation test, to remove random failures on FreeBSD.
Apparently there's still an occasional issue.
https://cirrus-ci.com/task/6613309985128448
result 232/352 (error): ERROR: duplicate key value violates unique constraint "ppln_uniqviol_pkey"
DETAIL: Key (id)=(116) already exists.
This shows that the issue is pretty rare:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/38/3511
--
Justin
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2022-06-27 09:29 Jelte Fennema <[email protected]>
parent: Justin Pryzby <[email protected]>
1 sibling, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-06-27 09:29 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Tom Lane <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
(resent because it was blocked from the mailing-list due to inclusion of a blocked email address in the To line)
From: Andres Freund <[email protected]>
> On 2022-04-04 15:21:54 +0000, Jelte Fennema wrote:
> > 2. Added some extra sleeps to the cancellation test, to remove random
> > failures on FreeBSD.
>
> That's extremely extremely rarely the solution to address test reliability
> issues. It'll fail when running test under valgrind etc.
>
> Why do you need sleeps / can you find another way to make the test reliable?
The problem they are solving is racy behaviour between sending the query
and sending the cancellation. If the cancellation is handled before the query
is started, then the query doesn't get cancelled. To solve this problem I used
the sleeps to wait a bit before sending the cancelation request.
When I wrote this, I couldn't think of a better way to do it then with sleeps.
But I didn't like it either (and I still don't). These emails made me start to think
again, about other ways of solving the problem. I think I've found another
solution (see attached patch). The way I solve it now is by using another
connection to check the state of the first one.
Jelte
Attachments:
[application/octet-stream] 0001-Add-documentation-for-libpq_pipeline-tests.patch (1.5K, ../../PR3PR83MB04765D30B272D65DE9E42CF1F7B99@PR3PR83MB0476.EURPRD83.prod.outlook.com/2-0001-Add-documentation-for-libpq_pipeline-tests.patch)
download | inline diff:
From 2be20b649243934d4fec1829d56fda7fc05b7268 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Thu, 13 Jan 2022 15:26:35 +0100
Subject: [PATCH 1/2] Add documentation for libpq_pipeline tests
This adds some explanation on how to run and add libpq tests.
---
src/test/modules/libpq_pipeline/README | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
index d8174dd579..6eda6c5756 100644
--- a/src/test/modules/libpq_pipeline/README
+++ b/src/test/modules/libpq_pipeline/README
@@ -1 +1,21 @@
Test programs and libraries for libpq
+=====================================
+
+You can manually run a specific test by running:
+
+ ./libpq_pipeline <name of test>
+
+To add a new libpq test to this module you need to edit libpq_pipeline.c. There
+you should add the name of your new test to the "print_test_list" function.
+Then in main you should do something when this test name is passed to the
+program.
+
+If the order in which postgres protocol messages are sent is deterministic for
+your test. Then you can generate a trace of these messages using the following
+command:
+
+ ./libpq_pipeline mynewtest -t traces/mynewtest.trace
+
+Once you've done that you should make sure that when running "make check"
+the generated trace is compared to the expected trace. This is done by adding
+your test name to the $cmptrace definition in the t/001_libpq_pipeline.pl file
--
2.34.1
[application/octet-stream] 0002-Add-non-blocking-version-of-PQcancel.patch (52.1K, ../../PR3PR83MB04765D30B272D65DE9E42CF1F7B99@PR3PR83MB0476.EURPRD83.prod.outlook.com/3-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 437b098b2ec6334affb2d818cf9154c7f170e2dc Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH 2/2] Add non-blocking version of PQcancel
This patch does four things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe but
blocking cancel (without having a need for signal-safety).
4. Use this new cancellation API everywhere in the codebase where
signal-safety is not a necessity.
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing
documentation for all the newly added functions.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQrequestCancelStart can now be used
instead, to have a non-blocking way of sending cancel requests. The
postgres_fdw cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 28 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 274 +++++++++++++-
15 files changed, 894 insertions(+), 153 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a561d1d652..b572cf6d5b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1379,22 +1379,30 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 061ffaf329..fa47274d15 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGconn *cancel_conn = PQrequestCancelStart(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQstatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGconn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQsocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQfinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQerrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQfinish(cancel_conn);
+ return failed;
}
+ PQfinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..6d108b56ef 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2567,6 +2567,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 92d1212027..7e02ed6803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -681,6 +682,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 37ec3cb4e5..8e033bb8f3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8856,10 +8988,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index f2e583f9fa..b9f0c0558c 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -158,19 +158,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 6e936bbff3..7390fbec7c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -379,6 +379,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -605,8 +606,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 8117cbd40f..c553a74898 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..b9ce1d58c1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..52503907bd 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,275 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ if (PQsendQuery(conn, "SELECT pg_sleep(30)") != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep(30)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1814,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1912,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-06-27 11:45 Justin Pryzby <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Justin Pryzby @ 2022-06-27 11:45 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
On Fri, Jun 24, 2022 at 07:36:16PM -0500, Justin Pryzby wrote:
> Resending with a problematic email removed from CC...
>
> On Mon, Apr 04, 2022 at 03:21:54PM +0000, Jelte Fennema wrote:
> > 2. Added some extra sleeps to the cancellation test, to remove random failures on FreeBSD.
>
> Apparently there's still an occasional issue.
> https://cirrus-ci.com/task/6613309985128448
I think that failure is actually not related to this patch.
There are probably others, but I noticed because it also affected one of my
patches, which changes nothing relevant.
https://cirrus-ci.com/task/5904044051922944
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-06-27 12:29 Alvaro Herrera <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Alvaro Herrera @ 2022-06-27 12:29 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Tom Lane <[email protected]>; Robert Haas <[email protected]>; Andres Freund <[email protected]>; [email protected]
On 2022-Jun-27, Justin Pryzby wrote:
> On Fri, Jun 24, 2022 at 07:36:16PM -0500, Justin Pryzby wrote:
> > Apparently there's still an occasional issue.
> > https://cirrus-ci.com/task/6613309985128448
>
> I think that failure is actually not related to this patch.
Yeah, it's not -- Kyotaro diagnosed it as a problem in libpq's pipeline
mode. I hope to push his fix soon, but there are nearby problems that I
haven't been able to track down a good fix for. I'm looking into the
whole.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2022-09-14 21:53 Tom Lane <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Tom Lane @ 2022-09-14 21:53 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Jelte Fennema <[email protected]> writes:
> [ non-blocking PQcancel ]
I pushed the 0001 patch (libpq_pipeline documentation) with a bit
of further wordsmithing.
As for 0002, I'm not sure that's anywhere near ready. I doubt it's
a great idea to un-deprecate PQrequestCancel with a major change
in its behavior. If there is anybody out there still using it,
they're not likely to appreciate that. Let's leave that alone and
pick some other name.
I'm also finding the entire design of PQrequestCancelStart etc to
be horribly confusing --- it's not *bad* necessarily, but the chosen
function names are seriously misleading. PQrequestCancelStart doesn't
actually "start" anything, so the apparent parallel with PQconnectStart
is just wrong. It's also fairly unclear what the state of a cancel
PQconn is after the request cycle is completed, and whether you can
re-use it (especially after a failed request), and whether you have
to dispose of it separately.
On the whole it feels like a mistake to have two separate kinds of
PGconn with fundamentally different behaviors and yet no distinction
in the API. I think I'd recommend having a separate struct type
(which might internally contain little more than a pointer to a
cloned PGconn), and provide only a limited set of operations on it.
Seems like create, start/continue cancel request, destroy, and
fetch error message ought to be enough. I don't see a reason why we
need to support all of libpq's inquiry operations on such objects ---
for instance, if you want to know which host is involved, you could
perfectly well query the parent PGconn. Nor do I want to run around
and add code to every single libpq entry point to make it reject cancel
PGconns if it can't support them, but we'd have to do so if there's
just one struct type.
I'm not seeing the use-case for PQconnectComplete. If you want
a non-blocking cancel request, why would you then use a blocking
operation to complete the request? Seems like it'd be better
to have just a monolithic cancel function for those who don't
need non-blocking.
This change:
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
is an absolute non-starter: it breaks ABI for every libpq client,
even ones that aren't using this facility. Why do we need a new
ConnStatusType value anyway? Seems like PostgresPollingStatusType
covers what we need: once you reach PGRES_POLLING_OK, the cancel
request is done.
The test case is still not very bulletproof on slow machines,
as it seems to be assuming that 30 seconds == forever. It
would be all right to use $PostgreSQL::Test::Utils::timeout_default,
but I'm not sure that that's easily retrievable by C code.
Maybe make the TAP test pass it in with another optional switch
to libpq_pipeline? Alternatively, we could teach libpq_pipeline
to do getenv("PG_TEST_TIMEOUT_DEFAULT") with a fallback to 180,
but that feels like it might be overly familiar with the innards
of Utils.pm.
regards, tom lane
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-10-05 13:23 Jelte Fennema <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-10-05 13:23 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Thanks for all the feedback. I attached a new patch that I think
addresses all of it. Below some additional info.
> On the whole it feels like a mistake to have two separate kinds of
> PGconn with fundamentally different behaviors and yet no distinction
> in the API. I think I'd recommend having a separate struct type
> (which might internally contain little more than a pointer to a
> cloned PGconn), and provide only a limited set of operations on it.
In my first version of this patch, this is exactly what I did. But then
I got this feedback from Jacob, so I changed it to reusing PGconn:
> > /* issue a cancel request */
> > extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
> > +extern PGcancelConn * PQcancelConnectStart(PGconn *conn);
> > +extern PGcancelConn * PQcancelConnect(PGconn *conn);
> > +extern PostgresPollingStatusType PQcancelConnectPoll(PGcancelConn * cancelConn);
> > +extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
> > +extern int PQcancelSocket(const PGcancelConn * cancelConn);
> > +extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
> > +extern void PQcancelFinish(PGcancelConn * cancelConn);
>
> That's a lot of new entry points, most of which don't do anything
> except call their twin after a pointer cast. How painful would it be to
> just use the existing APIs as-is, and error out when calling
> unsupported functions if conn->cancelRequest is true?
I changed it back to use PGcancelConn as per your suggestion and I
agree that the API got better because of it.
> + CONNECTION_CANCEL_FINISHED,
> /* Non-blocking mode only below here */
>
> is an absolute non-starter: it breaks ABI for every libpq client,
> even ones that aren't using this facility.
I removed this now. The main reason was so it was clear that no
queries could be sent over the connection, like is normally the case
when CONNECTION_OK happens. I don't think this is as useful anymore
now that this patch has a dedicated PGcancelStatus function.
NOTE: The CONNECTION_STARTING ConnStatusType is still necessary.
But to keep ABI compatibility I moved it to the end of the enum.
> Alternatively, we could teach libpq_pipeline
> to do getenv("PG_TEST_TIMEOUT_DEFAULT") with a fallback to 180,
> but that feels like it might be overly familiar with the innards
> of Utils.pm.
I went with this approach, because this environment variable was
already used in 2 other places than Utils.pm:
- contrib/test_decoding/sql/twophase.sql
- src/test/isolation/isolationtester.c
So, one more place seemed quite harmless.
P.S. I noticed a logical conflict between this patch and my libpq load
balancing patch. Because this patch depends on the connhost array
is constructed the exact same on the second invocation of connectOptions2.
But the libpq loadbalancing patch breaks this assumption. I'm making
a mental (and public) note that whichever of these patches gets merged last
should address this issue.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (52.0K, ../../DBBPR83MB0507788F756288750A6327F8F7469@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 437b098b2ec6334affb2d818cf9154c7f170e2dc Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH] Add non-blocking version of PQcancel
This patch does four things:
1. Change the PQrequestCancel implementation to use the regular
connection establishement code, to support all connection options
including encryption.
2. Add PQrequestCancelStart which is a thread-safe and non-blocking
version of this new PQrequestCancel implementation.
3. Add PQconnectComplete, which completes a connection started by
PQrequestCancelStart. This is useful if you want a thread-safe but
blocking cancel (without having a need for signal-safety).
4. Use this new cancellation API everywhere in the codebase where
signal-safety is not a necessity.
This change un-deprecates PQrequestCancel, since now there's actually an
advantage to using it over PQcancel. It also includes user facing
documentation for all the newly added functions.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQrequestCancelStart can now be used
instead, to have a non-blocking way of sending cancel requests. The
postgres_fdw cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 28 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 212 +++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 2 +
src/interfaces/libpq/fe-connect.c | 341 +++++++++++++++---
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 9 +-
src/interfaces/libpq/libpq-int.h | 4 +
src/test/isolation/isolationtester.c | 28 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 274 +++++++++++++-
15 files changed, 894 insertions(+), 153 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a561d1d652..b572cf6d5b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1379,22 +1379,30 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGconn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
+ }
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQconnectComplete(cancelConn))
+ {
+ msg = "OK";
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = pchomp(PQerrorMessage(cancelConn));
+ }
+ PQfinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 061ffaf329..fa47274d15 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGconn *cancel_conn = PQrequestCancelStart(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQstatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQerrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGconn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQsocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQfinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQerrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQfinish(cancel_conn);
+ return failed;
}
+ PQfinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..6d108b56ef 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2567,6 +2567,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 92d1212027..7e02ed6803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -681,6 +682,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 37ec3cb4e5..8e033bb8f3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -499,6 +499,30 @@ switch(PQstatus(conn))
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQconnectComplete">
+ <term><function>PQconnectComplete</function><indexterm><primary>PQconnectComplete</primary></indexterm></term>
+ <listitem>
+ <para>
+ Complete the connection attempt on a nonblocking connection and block
+ until it is completed.
+
+<synopsis>
+int PQconnectPoll(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function can be used instead of
+ <xref linkend="libpq-PQconnectPoll"/>
+ to complete a connection that was initially started in a non blocking
+ manner. However, instead of continuing to complete the connection in a
+ non blocking way, calling this function will block until the connection
+ is completed. This is especially useful to complete connections that were
+ started by <xref linkend="libpq-PQrequestCancelStart"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQconndefaults">
<term><function>PQconndefaults</function><indexterm><primary>PQconndefaults</primary></indexterm></term>
<listitem>
@@ -660,7 +684,7 @@ void PQreset(PGconn *conn);
<varlistentry id="libpq-PQresetStart">
<term><function>PQresetStart</function><indexterm><primary>PQresetStart</primary></indexterm></term>
- <term><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
+ <term id="libpq-PQresetPoll"><function>PQresetPoll</function><indexterm><primary>PQresetPoll</primary></indexterm></term>
<listitem>
<para>
Reset the communication channel to the server, in a nonblocking manner.
@@ -5617,13 +5641,137 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQrequestCancel">
+ <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+int PQrequestCancel(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function operates directly on the <structname>PGconn</structname>
+ object, and in case of failure stores the error message in the
+ <structname>PGconn</structname> object (whence it can be retrieved
+ by <xref linkend="libpq-PQerrorMessage"/>). This behaviour makes this
+ function unsafe to call from within multi-threaded programs or
+ signal handlers, since it is possible that overwriting the
+ <structname>PGconn</structname>'s error message will
+ mess up the operation currently in progress on the connection in another
+ thread.
+ </para>
+
+ <para>
+ The return value is 1 if the cancel request was successfully
+ dispatched and 0 if not. Successful dispatch is no guarantee that the
+ request will have any effect, however. If the cancellation is effective,
+ the current command will terminate early and return an error result. If
+ the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at
+ all.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQrequestCancelStart">
+ <term><function>PQrequestCancelStart</function><indexterm><primary>PQrequestCancelStart</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used in thread-safe and/or non-blocking manner.
+<synopsis>
+PGconn *PQrequestCancelStart(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This function returns a new <structname>PGconn</structname>. This
+ connection object can be used to cancel the query that's running on the
+ original connection in a thread-safe way. To do so
+ <xref linkend="libpq-PQrequestCancel"/>
+ must be called while no other thread is using the original PGconn. Then
+ the returned <structname>PGconn</structname>
+ can be used at a later point in any thread to send a cancel request.
+ A cancel request can be sent using the returned PGconn in two ways,
+ non-blocking using <xref linkend="libpq-PQconnectPoll"/>
+ or blocking using <xref linkend="libpq-PQconnectComplete"/>.
+ </para>
+
+ <para>
+ In addition to all the statuses that a regular
+ <structname>PGconn</structname>
+ can have returned connection can have two additional statuses:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQconnectPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQrequestCancel"/>. No connection to the
+ server has been initiated yet at this point. To start cancel request
+ initiation use <xref linkend="libpq-PQconnectPoll"/>
+ for non-blocking behaviour and <xref linkend="libpq-PQconnectComplete"/>
+ for blocking behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connection-cancel-finished">
+ <term><symbol>CONNECTION_CANCEL_FINISHED</symbol></term>
+ <listitem>
+ <para>
+ Cancel request was successfully sent. It's not possible to continue
+ using the cancellation connection now, so it should be freed using
+ <xref linkend="libpq-PQfinish"/>. It's also possible to reset the
+ cancellation connection instead using
+ <xref linkend="libpq-PQresetStart"/>, that way it can be reused to
+ cancel a future query on the same connection.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Since this object represents a connection only meant for cancellations it
+ can only be used with a limited subset of the functions that can be used
+ for a regular <structname>PGconn</structname> object. The functions that
+ this object can be passed to are
+ <xref linkend="libpq-PQstatus"/>,
+ <xref linkend="libpq-PQerrorMessage"/>,
+ <xref linkend="libpq-PQconnectComplete"/>,
+ <xref linkend="libpq-PQconnectPoll"/>,
+ <xref linkend="libpq-PQsocket"/>,
+ <xref linkend="libpq-PQresetStart"/>, and
+ <xref linkend="libpq-PQfinish"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5665,7 +5813,9 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ A less secure version of
+ <xref linkend="libpq-PQrequestCancel"/>
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
@@ -5679,15 +5829,6 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
recommended size is 256 bytes).
</para>
- <para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
<para>
<xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
handler, if the <parameter>errbuf</parameter> is a local variable in the
@@ -5696,33 +5837,24 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
also be invoked from a thread that is separate from the one
manipulating the <structname>PGconn</structname> object.
</para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <variablelist>
- <varlistentry id="libpq-PQrequestCancel">
- <term><function>PQrequestCancel</function><indexterm><primary>PQrequestCancel</primary></indexterm></term>
-
- <listitem>
- <para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
-<synopsis>
-int PQrequestCancel(PGconn *conn);
-</synopsis>
- </para>
<para>
- Requests that the server abandon processing of the current
- command. It operates directly on the
- <structname>PGconn</structname> object, and in case of failure stores the
- error message in the <structname>PGconn</structname> object (whence it can
- be retrieved by <xref linkend="libpq-PQerrorMessage"/>). Although
- the functionality is the same, this approach is not safe within
- multiple-thread programs or signal handlers, since it is possible
- that overwriting the <structname>PGconn</structname>'s error message will
- mess up the operation currently in progress on the connection.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. This means the connection
+ is never encrypted using TLS or GSS.
</para>
</listitem>
</varlistentry>
@@ -8856,10 +8988,10 @@ int PQisthreadsafe();
</para>
<para>
- The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
+ The functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQrequestCancelStart"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index f2e583f9fa..b9f0c0558c 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -158,19 +158,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQrequestCancel(conn);
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f7609d0c64 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,5 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQrequestCancelStart 187
+PQconnectComplete 188
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 6e936bbff3..7390fbec7c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -379,6 +379,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -605,8 +606,17 @@ pqDropServerData(PGconn *conn)
if (conn->write_err_msg)
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -737,6 +747,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConnectStart
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGconn *
+PQrequestCancelStart(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGconn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -914,6 +986,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2082,10 +2194,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConnectStart. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2134,6 +2253,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2274,13 +2402,15 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
case CONNECTION_OK:
return PGRES_POLLING_OK;
+ case CONNECTION_CANCEL_FINISHED:
+ return PGRES_POLLING_OK;
/* These are reading states */
case CONNECTION_AWAITING_RESPONSE:
@@ -2292,6 +2422,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_CANCEL_FINISHED;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2301,6 +2459,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2325,6 +2484,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2344,6 +2511,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2529,19 +2705,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConnectStart.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2557,7 +2741,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2567,12 +2751,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2595,7 +2785,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2624,7 +2814,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2718,8 +2908,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2758,6 +2949,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2966,6 +3167,25 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4194,6 +4414,11 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4311,6 +4536,12 @@ PQresetStart(PGconn *conn)
{
closePGconn(conn);
+ if (conn->cancelRequest)
+ {
+ conn->status = CONNECTION_STARTING;
+ return 1;
+ }
+
return connectDBStart(conn);
}
@@ -4663,6 +4894,22 @@ cancel_errReturn:
return false;
}
+/*
+ * PQconnectComplete: takes a non blocking cancel connection and completes it
+ * in a blocking manner.
+ *
+ * Returns 1 if able to connect successfully and 0 if not.
+ *
+ * This can useful if you only care about the thread safety of
+ * PQrequestCancelStart and not about its non blocking functionality.
+ */
+int
+PQconnectComplete(PGconn *cancelConn)
+{
+ connectDBComplete(cancelConn);
+ return cancelConn->status != CONNECTION_BAD;
+}
+
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4679,45 +4926,31 @@ cancel_errReturn:
int
PQrequestCancel(PGconn *conn)
{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
+ PGconn *cancelConn = NULL;
- if (conn->sock == PGINVALID_SOCKET)
+ cancelConn = PQrequestCancelStart(conn);
+ if (!cancelConn)
{
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext("out of memory\n"));
return false;
}
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
+ if (cancelConn->status == CONNECTION_BAD)
{
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- if (!r)
+ if (!PQconnectComplete(cancelConn))
{
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
+ appendPQExpBufferStr(&conn->errorMessage, PQerrorMessage(cancelConn));
+ freePGconn(cancelConn);
+ return false;
}
- return r;
+ freePGconn(cancelConn);
+ return true;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index d76bb3957a..a944cb2c12 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -558,8 +558,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -642,7 +645,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -737,7 +740,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -755,13 +758,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 8117cbd40f..c553a74898 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -255,7 +255,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index a1dc7b796d..9771805dd3 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -201,6 +201,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is return.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..24695a6026 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -59,12 +59,15 @@ typedef enum
{
CONNECTION_OK,
CONNECTION_BAD,
+ CONNECTION_CANCEL_FINISHED,
/* Non-blocking mode only below here */
/*
* The existence of these should never be relied upon - they should only
* be used for user feedback or similar purposes.
*/
+ CONNECTION_STARTING, /* Waiting for connection attempt to be
+ * started. */
CONNECTION_STARTED, /* Waiting for connection to be made. */
CONNECTION_MADE, /* Connection OK; waiting to send. */
CONNECTION_AWAITING_RESPONSE, /* Waiting for a response from the
@@ -282,6 +285,7 @@ extern PGconn *PQconnectStart(const char *conninfo);
extern PGconn *PQconnectStartParams(const char *const *keywords,
const char *const *values, int expand_dbname);
extern PostgresPollingStatusType PQconnectPoll(PGconn *conn);
+extern int PQconnectComplete(PGconn *conn);
/* Synchronous (blocking) */
extern PGconn *PQconnectdb(const char *conninfo);
@@ -330,9 +334,12 @@ extern void PQfreeCancel(PGcancel *cancel);
/* issue a cancel request */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* more secure version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
+/* non-blocking and thread-safe version of PQrequestCancel */
+extern PGconn *PQrequestCancelStart(PGconn *conn);
+
/* Accessor functions for PGconn objects */
extern char *PQdb(const PGconn *conn);
extern char *PQuser(const PGconn *conn);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..b9ce1d58c1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 12179f2514..b073235197 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -948,26 +948,18 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
-
- if (cancel != NULL)
+ if (PQrequestCancel(conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQerrorMessage(conn));
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 0ff563f59a..52503907bd 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,275 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ if (PQsendQuery(conn, "SELECT pg_sleep(30)") != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep(30)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGconn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQrequestCancelStart and then polling with PQcancelConnectPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQconnectPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQresetStart works on the cancel connection and it can be reused
+ * after
+ */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQresetPoll(cancelConn);
+ int sock = PQsocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQerrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQstatus(cancelConn) != CONNECTION_CANCEL_FINISHED)
+ pg_fatal("unexpected cancel connection status: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQfinish(cancelConn);
+
+ /* test PQconnectComplete */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQrequestCancelStart(conn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /* test PQconnectComplete with reset connection */
+ if (!PQresetStart(cancelConn))
+ {
+ pg_fatal("cancel connection reset failed: %s", PQerrorMessage(cancelConn));
+ }
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQstatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQerrorMessage(cancelConn));
+ if (!PQconnectComplete(cancelConn))
+ pg_fatal("failed to send cancel: %s", PQerrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQfinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1545,6 +1814,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1642,7 +1912,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-11-04 15:58 Jacob Champion <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jacob Champion @ 2022-11-04 15:58 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 10/5/22 06:23, Jelte Fennema wrote:
> In my first version of this patch, this is exactly what I did. But then
> I got this feedback from Jacob, so I changed it to reusing PGconn:
>
>> [snip]
>
> I changed it back to use PGcancelConn as per your suggestion and I
> agree that the API got better because of it.
Sorry for the whiplash!
Is the latest attachment the correct version? I don't see any difference
between the latest 0001 and the previous version's 0002 -- it has no
references to PG_TEST_TIMEOUT_DEFAULT, PGcancelConn, etc.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-11-15 11:38 Jelte Fennema <[email protected]>
parent: Jacob Champion <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema @ 2022-11-15 11:38 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Ugh, it indeed seems like I somehow messed up sending the new patch.
Here's the correct one.
Attachments:
[application/octet-stream] 0001-Add-non-blocking-version-of-PQcancel.patch (55.8K, ../../DBBPR83MB0507E785FE96166AB8D9940BF7049@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-0001-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From d8d581a0033e0365faf96a39e8ce75a6ec9ebf7d Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
3. Use these two new cancellation APIs everywhere in the codebase where
signal-safety is not a necessity.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests. The postgres_fdw
cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 22 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 279 +++++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 375 ++++++++++++++++--
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 25 +-
src/interfaces/libpq/libpq-int.h | 9 +
src/test/isolation/isolationtester.c | 29 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++-
15 files changed, 1050 insertions(+), 109 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 9eef417c47..2a55c6759a 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1378,22 +1378,24 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQcancelSend(conn);
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = "OK";
+ }
+ PQcancelFinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 939d114f02..9622441da7 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQcancelFinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQcancelFinish(cancel_conn);
+ return failed;
}
+ PQcancelFinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index cc9e39c4a5..113f3204cc 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2688,6 +2688,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index e48ccd286b..bf977442d6 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -713,6 +714,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 3c9bd3d673..90db021c1d 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -4909,7 +4909,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -5627,13 +5627,220 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+PGcancelConn *PQcancelSend(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function returns a <structname>PGcancelConn</structname>
+ object. By using
+ <xref linkend="libpq-PQcancelStatus"/>
+ it can be checked if there was any error when sending the cancellation
+ request. If <xref linkend="libpq-PQcancelStatus"/>
+ returns for <symbol>CONNECTION_OK</symbol> the request was
+ successfully sent, but if it returns <symbol>CONNECTION_BAD</symbol>
+ an error occured. If an error occured the error message can be retrieved using
+ <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being cancelled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelSend</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQcancelSend"/> that can be used
+ in a non-blocking manner.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>,
+ but it won't instantly start sending a cancel request over this
+ connection like <xref linkend="libpq-PQcancelSend"/>.
+ <xref linkend="libpq-PQcancelStatus"/> should be called on the return
+ value to check if the <structname> PGcancelConn </structname> was
+ created successfully.
+ The <structname>PGcancelConn</structname> object is an opaque structure
+ that is not meant to be accessed directly by the application.
+ This <structname>PGcancelConn</structname> object can be used to cancel
+ the query that's running on the original connection in a thread-safe and
+ non-blocking way.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually cancelled).
+ While a <symbol>CONNECTION_OK</symbol> result for
+ <structname>PGconn</structname> means thatqueries can be sent over the
+ connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5675,14 +5882,30 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of
+ <xref linkend="libpq-PQcancelSend"/>, but one that can be used safely
+ from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead.
+ <xref linkend="libpq-PQcancel"/> can be safely invoked from a signal
+ handler, if the <parameter>errbuf</parameter> is a local variable in the
+ signal handler. The <structname>PGcancel</structname> object is read-only
+ as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
+ also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -5690,21 +5913,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -5716,13 +5940,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -8871,7 +9104,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 1cc97b72f7..0f5e84ad71 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ PQcancelFinish(PQcancelSend(conn));
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f56e8c185c 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,11 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelSend 187
+PQcancelConn 188
+PQcancelPoll 189
+PQcancelStatus 190
+PQcancelSocket 191
+PQcancelErrorMessage 192
+PQcancelReset 193
+PQcancelFinish 194
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..7b59697e64 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -376,6 +376,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -599,8 +600,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -731,6 +741,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection was NULL\n"));
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ appendPQExpBufferStr(&cancelConn->errorMessage, libpq_gettext("passed connection is not open\n"));
+ return (PGcancelConn *) cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -907,6 +979,46 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ appendPQExpBufferStr(&dstConn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2055,10 +2167,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConn. So leave these settings
+ * alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2107,6 +2226,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2247,8 +2375,8 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
@@ -2265,6 +2393,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2274,6 +2430,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2298,6 +2455,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2317,6 +2482,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2498,19 +2672,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConn.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2526,7 +2708,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2536,12 +2718,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
appendPQExpBuffer(&conn->errorMessage,
@@ -2564,7 +2752,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2593,7 +2781,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2687,8 +2875,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2727,6 +2916,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2935,6 +3134,30 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancelation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not send cancel packet: %s\n"),
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4114,6 +4337,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a Terminate
+ * message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4582,6 +4814,96 @@ cancel_errReturn:
return false;
}
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+PGcancelConn *
+PQcancelSend(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConn(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn *cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4640,6 +4962,7 @@ PQrequestCancel(PGconn *conn)
}
+
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 795500c593..b5b10ec2ba 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -556,8 +556,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -640,7 +643,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -735,7 +738,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -753,13 +756,17 @@ definitelyEOF:
libpq_gettext("server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.\n"));
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index b42a908733..ca378d2ad5 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -253,7 +253,7 @@ rloop:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("SSL connection has been closed unexpectedly\n"));
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
appendPQExpBuffer(&conn->errorMessage,
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 3df4a97f2e..18dff253c4 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -199,6 +199,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is returned.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index b7df3224c0..de2e32ca63 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,28 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* issue a cancel request */
+extern PGcancelConn * PQcancelSend(PGconn *conn);
+/* non-blocking version of PQrequestSend */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn *cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..84027bc4ab 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -397,6 +397,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -592,6 +596,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153..3781f7982b 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelSend(conn);
- if (cancel != NULL)
+ if (PQcancelStatus(cancel_conn) == CONNECTION_OK)
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index c609f42258..2674abb539 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelSend(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1638,6 +1896,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1739,7 +1998,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: Add non-blocking version of PQcancel
@ 2022-11-29 19:17 Daniel Gustafsson <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Daniel Gustafsson @ 2022-11-29 19:17 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
> On 15 Nov 2022, at 12:38, Jelte Fennema <[email protected]> wrote:
> Here's the correct one.<0001-Add-non-blocking-version-of-PQcancel.patch>
This version of the patch no longer applies, a rebased version is needed.
--
Daniel Gustafsson https://vmware.com/
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2022-11-30 09:20 Jelte Fennema <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Jelte Fennema @ 2022-11-30 09:20 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Jacob Champion <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
> This version of the patch no longer applies, a rebased version is needed.
Attached is a patch that applies cleanly again and is also changed
to use the recently introduced libpq_append_conn_error.
I also attached a patch that runs pgindent after the introduction of
libpq_append_conn_error. I noticed that this hadn't happened when
trying to run pgindent on my own changes.
Attachments:
[application/octet-stream] v9-0001-libpq-Run-pgindent-after-a9e9a9f32b3.patch (39.3K, ../../DBBPR83MB05071BB8A8838830CABAFFE1F7129@DBBPR83MB0507.EURPRD83.prod.outlook.com/2-v9-0001-libpq-Run-pgindent-after-a9e9a9f32b3.patch)
download | inline diff:
From 9bf997762786f9d276f211c588918a1cdca598d5 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 30 Nov 2022 10:07:19 +0100
Subject: [PATCH v9 1/2] libpq: Run pgindent after a9e9a9f32b3
It seems that pgindent was not run after the error handling refactor in
commit a9e9a9f32b35edf129c88e8b929ef223f8511f59. This fixes that and
also addresses a few other things pgindent wanted to change in libpq.
---
src/interfaces/libpq/fe-auth-scram.c | 2 +-
src/interfaces/libpq/fe-auth.c | 8 +-
src/interfaces/libpq/fe-connect.c | 124 +++++++++++------------
src/interfaces/libpq/fe-exec.c | 16 +--
src/interfaces/libpq/fe-lobj.c | 42 ++++----
src/interfaces/libpq/fe-misc.c | 10 +-
src/interfaces/libpq/fe-protocol3.c | 2 +-
src/interfaces/libpq/fe-secure-common.c | 6 +-
src/interfaces/libpq/fe-secure-gssapi.c | 12 +--
src/interfaces/libpq/fe-secure-openssl.c | 64 ++++++------
src/interfaces/libpq/fe-secure.c | 8 +-
src/interfaces/libpq/libpq-int.h | 4 +-
12 files changed, 149 insertions(+), 149 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index e71626580a..b8e961795d 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -710,7 +710,7 @@ read_server_final_message(fe_scram_state *state, char *input)
return false;
}
libpq_append_conn_error(conn, "error received from server in SCRAM exchange: %s",
- errmsg);
+ errmsg);
return false;
}
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 4a6c358bb6..3caca455aa 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -73,7 +73,7 @@ pg_GSS_continue(PGconn *conn, int payloadlen)
if (!ginbuf.value)
{
libpq_append_conn_error(conn, "out of memory allocating GSSAPI buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
if (pqGetnchar(ginbuf.value, payloadlen, conn))
@@ -223,7 +223,7 @@ pg_SSPI_continue(PGconn *conn, int payloadlen)
if (!inputbuf)
{
libpq_append_conn_error(conn, "out of memory allocating SSPI buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
if (pqGetnchar(inputbuf, payloadlen, conn))
@@ -623,7 +623,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
if (!challenge)
{
libpq_append_conn_error(conn, "out of memory allocating SASL buffer (%d)",
- payloadlen);
+ payloadlen);
return STATUS_ERROR;
}
@@ -1277,7 +1277,7 @@ PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user,
else
{
libpq_append_conn_error(conn, "unrecognized password encryption algorithm \"%s\"",
- algorithm);
+ algorithm);
return NULL;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f88d672c6c..d0c3b21fb9 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -1079,7 +1079,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "could not match %d host names to %d hostaddr values",
- count_comma_separated_elems(conn->pghost), conn->nconnhost);
+ count_comma_separated_elems(conn->pghost), conn->nconnhost);
return false;
}
}
@@ -1159,7 +1159,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "could not match %d port numbers to %d hosts",
- count_comma_separated_elems(conn->pgport), conn->nconnhost);
+ count_comma_separated_elems(conn->pgport), conn->nconnhost);
return false;
}
}
@@ -1248,7 +1248,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "channel_binding", conn->channel_binding);
+ "channel_binding", conn->channel_binding);
return false;
}
}
@@ -1273,7 +1273,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "sslmode", conn->sslmode);
+ "sslmode", conn->sslmode);
return false;
}
@@ -1293,7 +1293,7 @@ connectOptions2(PGconn *conn)
case 'v': /* "verify-ca" or "verify-full" */
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "sslmode value \"%s\" invalid when SSL support is not compiled in",
- conn->sslmode);
+ conn->sslmode);
return false;
}
#endif
@@ -1313,16 +1313,16 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "ssl_min_protocol_version",
- conn->ssl_min_protocol_version);
+ "ssl_min_protocol_version",
+ conn->ssl_min_protocol_version);
return false;
}
if (!sslVerifyProtocolVersion(conn->ssl_max_protocol_version))
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "ssl_max_protocol_version",
- conn->ssl_max_protocol_version);
+ "ssl_max_protocol_version",
+ conn->ssl_max_protocol_version);
return false;
}
@@ -1359,7 +1359,7 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when GSSAPI support is not compiled in",
- conn->gssencmode);
+ conn->gssencmode);
return false;
}
#endif
@@ -1392,8 +1392,8 @@ connectOptions2(PGconn *conn)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
- "target_session_attrs",
- conn->target_session_attrs);
+ "target_session_attrs",
+ conn->target_session_attrs);
return false;
}
}
@@ -1609,7 +1609,7 @@ connectNoDelay(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "could not set socket to TCP no delay mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1787,7 +1787,7 @@ parse_int_param(const char *value, int *result, PGconn *conn,
error:
libpq_append_conn_error(conn, "invalid integer value \"%s\" for connection option \"%s\"",
- value, context);
+ value, context);
return false;
}
@@ -1816,9 +1816,9 @@ setKeepalivesIdle(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- PG_TCP_KEEPALIVE_IDLE_STR,
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ PG_TCP_KEEPALIVE_IDLE_STR,
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1850,9 +1850,9 @@ setKeepalivesInterval(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_KEEPINTVL",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_KEEPINTVL",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1885,9 +1885,9 @@ setKeepalivesCount(PGconn *conn)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_KEEPCNT",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_KEEPCNT",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -1949,8 +1949,8 @@ prepKeepalivesWin32(PGconn *conn)
if (!setKeepalivesWin32(conn->sock, idle, interval))
{
libpq_append_conn_error(conn, "%s(%s) failed: error code %d",
- "WSAIoctl", "SIO_KEEPALIVE_VALS",
- WSAGetLastError());
+ "WSAIoctl", "SIO_KEEPALIVE_VALS",
+ WSAGetLastError());
return 0;
}
return 1;
@@ -1983,9 +1983,9 @@ setTCPUserTimeout(PGconn *conn)
char sebuf[256];
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "TCP_USER_TIMEOUT",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "TCP_USER_TIMEOUT",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
return 0;
}
#endif
@@ -2354,7 +2354,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not translate host name \"%s\" to address: %s",
- ch->host, gai_strerror(ret));
+ ch->host, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2366,7 +2366,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not parse network address \"%s\": %s",
- ch->hostaddr, gai_strerror(ret));
+ ch->hostaddr, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2377,8 +2377,8 @@ keep_going: /* We will come back to here until there is
if (strlen(portstr) >= UNIXSOCK_PATH_BUFLEN)
{
libpq_append_conn_error(conn, "Unix-domain socket path \"%s\" is too long (maximum %d bytes)",
- portstr,
- (int) (UNIXSOCK_PATH_BUFLEN - 1));
+ portstr,
+ (int) (UNIXSOCK_PATH_BUFLEN - 1));
goto keep_going;
}
@@ -2391,7 +2391,7 @@ keep_going: /* We will come back to here until there is
if (ret || !conn->addrlist)
{
libpq_append_conn_error(conn, "could not translate Unix-domain socket path \"%s\" to address: %s",
- portstr, gai_strerror(ret));
+ portstr, gai_strerror(ret));
goto keep_going;
}
break;
@@ -2513,7 +2513,7 @@ keep_going: /* We will come back to here until there is
}
emitHostIdentityInfo(conn, host_addr);
libpq_append_conn_error(conn, "could not create socket: %s",
- SOCK_STRERROR(errorno, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(errorno, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2543,7 +2543,7 @@ keep_going: /* We will come back to here until there is
if (!pg_set_noblock(conn->sock))
{
libpq_append_conn_error(conn, "could not set socket to nonblocking mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
conn->try_next_addr = true;
goto keep_going;
}
@@ -2552,7 +2552,7 @@ keep_going: /* We will come back to here until there is
if (fcntl(conn->sock, F_SETFD, FD_CLOEXEC) == -1)
{
libpq_append_conn_error(conn, "could not set socket to close-on-exec mode: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
conn->try_next_addr = true;
goto keep_going;
}
@@ -2581,9 +2581,9 @@ keep_going: /* We will come back to here until there is
(char *) &on, sizeof(on)) < 0)
{
libpq_append_conn_error(conn, "%s(%s) failed: %s",
- "setsockopt",
- "SO_KEEPALIVE",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ "setsockopt",
+ "SO_KEEPALIVE",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
err = 1;
}
else if (!setKeepalivesIdle(conn)
@@ -2708,7 +2708,7 @@ keep_going: /* We will come back to here until there is
(char *) &optval, &optlen) == -1)
{
libpq_append_conn_error(conn, "could not get socket error status: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
else if (optval != 0)
@@ -2735,7 +2735,7 @@ keep_going: /* We will come back to here until there is
&conn->laddr.salen) < 0)
{
libpq_append_conn_error(conn, "could not get client address from socket: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2775,7 +2775,7 @@ keep_going: /* We will come back to here until there is
libpq_append_conn_error(conn, "requirepeer parameter is not supported on this platform");
else
libpq_append_conn_error(conn, "could not get peer credentials: %s",
- strerror_r(errno, sebuf, sizeof(sebuf)));
+ strerror_r(errno, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2788,7 +2788,7 @@ keep_going: /* We will come back to here until there is
if (strcmp(remote_username, conn->requirepeer) != 0)
{
libpq_append_conn_error(conn, "requirepeer specifies \"%s\", but actual peer user name is \"%s\"",
- conn->requirepeer, remote_username);
+ conn->requirepeer, remote_username);
free(remote_username);
goto error_return;
}
@@ -2829,7 +2829,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send GSSAPI negotiation packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
@@ -2840,7 +2840,7 @@ keep_going: /* We will come back to here until there is
else if (!conn->gctx && conn->gssencmode[0] == 'r')
{
libpq_append_conn_error(conn,
- "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
+ "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
goto error_return;
}
#endif
@@ -2882,7 +2882,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send SSL negotiation packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
goto error_return;
}
/* Ok, wait for response */
@@ -2911,7 +2911,7 @@ keep_going: /* We will come back to here until there is
if (pqPacketSend(conn, 0, startpacket, packetlen) != STATUS_OK)
{
libpq_append_conn_error(conn, "could not send startup packet: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
free(startpacket);
goto error_return;
}
@@ -3012,7 +3012,7 @@ keep_going: /* We will come back to here until there is
else
{
libpq_append_conn_error(conn, "received invalid response to SSL negotiation: %c",
- SSLok);
+ SSLok);
goto error_return;
}
}
@@ -3123,7 +3123,7 @@ keep_going: /* We will come back to here until there is
else if (gss_ok != 'G')
{
libpq_append_conn_error(conn, "received invalid response to GSSAPI negotiation: %c",
- gss_ok);
+ gss_ok);
goto error_return;
}
}
@@ -3201,7 +3201,7 @@ keep_going: /* We will come back to here until there is
if (!(beresp == 'R' || beresp == 'v' || beresp == 'E'))
{
libpq_append_conn_error(conn, "expected authentication request from server, but received %c",
- beresp);
+ beresp);
goto error_return;
}
@@ -3216,17 +3216,17 @@ keep_going: /* We will come back to here until there is
* Try to validate message length before using it.
* Authentication requests can't be very large, although GSS
* auth requests may not be that small. Same for
- * NegotiateProtocolVersion. Errors can be a
- * little larger, but not huge. If we see a large apparent
- * length in an error, it means we're really talking to a
- * pre-3.0-protocol server; cope. (Before version 14, the
- * server also used the old protocol for errors that happened
- * before processing the startup packet.)
+ * NegotiateProtocolVersion. Errors can be a little larger,
+ * but not huge. If we see a large apparent length in an
+ * error, it means we're really talking to a pre-3.0-protocol
+ * server; cope. (Before version 14, the server also used the
+ * old protocol for errors that happened before processing the
+ * startup packet.)
*/
if ((beresp == 'R' || beresp == 'v') && (msgLength < 8 || msgLength > 2000))
{
libpq_append_conn_error(conn, "expected authentication request from server, but received %c",
- beresp);
+ beresp);
goto error_return;
}
@@ -3705,7 +3705,7 @@ keep_going: /* We will come back to here until there is
/* Append error report to conn->errorMessage. */
libpq_append_conn_error(conn, "\"%s\" failed",
- "SHOW transaction_read_only");
+ "SHOW transaction_read_only");
/* Close connection politely. */
conn->status = CONNECTION_OK;
@@ -3755,7 +3755,7 @@ keep_going: /* We will come back to here until there is
/* Append error report to conn->errorMessage. */
libpq_append_conn_error(conn, "\"%s\" failed",
- "SELECT pg_is_in_recovery()");
+ "SELECT pg_is_in_recovery()");
/* Close connection politely. */
conn->status = CONNECTION_OK;
@@ -3768,8 +3768,8 @@ keep_going: /* We will come back to here until there is
default:
libpq_append_conn_error(conn,
- "invalid connection state %d, probably indicative of memory corruption",
- conn->status);
+ "invalid connection state %d, probably indicative of memory corruption",
+ conn->status);
goto error_return;
}
@@ -7148,7 +7148,7 @@ pgpassfileWarning(PGconn *conn)
if (sqlstate && strcmp(sqlstate, ERRCODE_INVALID_PASSWORD) == 0)
libpq_append_conn_error(conn, "password retrieved from file \"%s\"",
- conn->pgpassfile);
+ conn->pgpassfile);
}
}
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index da229d632a..88600ce883 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -1444,7 +1444,7 @@ PQsendQueryInternal(PGconn *conn, const char *query, bool newQuery)
if (conn->pipelineStatus != PQ_PIPELINE_OFF)
{
libpq_append_conn_error(conn, "%s not allowed in pipeline mode",
- "PQsendQuery");
+ "PQsendQuery");
return 0;
}
@@ -1512,7 +1512,7 @@ PQsendQueryParams(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -1558,7 +1558,7 @@ PQsendPrepare(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -1652,7 +1652,7 @@ PQsendQueryPrepared(PGconn *conn,
if (nParams < 0 || nParams > PQ_QUERY_PARAM_MAX_LIMIT)
{
libpq_append_conn_error(conn, "number of parameters must be between 0 and %d",
- PQ_QUERY_PARAM_MAX_LIMIT);
+ PQ_QUERY_PARAM_MAX_LIMIT);
return 0;
}
@@ -2099,10 +2099,9 @@ PQgetResult(PGconn *conn)
/*
* We're about to return the NULL that terminates the round of
- * results from the current query; prepare to send the results
- * of the next query, if any, when we're called next. If there's
- * no next element in the command queue, this gets us in IDLE
- * state.
+ * results from the current query; prepare to send the results of
+ * the next query, if any, when we're called next. If there's no
+ * next element in the command queue, this gets us in IDLE state.
*/
pqPipelineProcessQueue(conn);
res = NULL; /* query is complete */
@@ -3047,6 +3046,7 @@ pqPipelineProcessQueue(PGconn *conn)
return;
case PGASYNC_IDLE:
+
/*
* If we're in IDLE mode and there's some command in the queue,
* get us into PIPELINE_IDLE mode and process normally. Otherwise
diff --git a/src/interfaces/libpq/fe-lobj.c b/src/interfaces/libpq/fe-lobj.c
index bcd228cef1..50282ff423 100644
--- a/src/interfaces/libpq/fe-lobj.c
+++ b/src/interfaces/libpq/fe-lobj.c
@@ -142,7 +142,7 @@ lo_truncate(PGconn *conn, int fd, size_t len)
if (conn->lobjfuncs->fn_lo_truncate == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_truncate");
+ "lo_truncate");
return -1;
}
@@ -205,7 +205,7 @@ lo_truncate64(PGconn *conn, int fd, pg_int64 len)
if (conn->lobjfuncs->fn_lo_truncate64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_truncate64");
+ "lo_truncate64");
return -1;
}
@@ -395,7 +395,7 @@ lo_lseek64(PGconn *conn, int fd, pg_int64 offset, int whence)
if (conn->lobjfuncs->fn_lo_lseek64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_lseek64");
+ "lo_lseek64");
return -1;
}
@@ -485,7 +485,7 @@ lo_create(PGconn *conn, Oid lobjId)
if (conn->lobjfuncs->fn_lo_create == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_create");
+ "lo_create");
return InvalidOid;
}
@@ -558,7 +558,7 @@ lo_tell64(PGconn *conn, int fd)
if (conn->lobjfuncs->fn_lo_tell64 == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_tell64");
+ "lo_tell64");
return -1;
}
@@ -667,7 +667,7 @@ lo_import_internal(PGconn *conn, const char *filename, Oid oid)
if (fd < 0)
{ /* error */
libpq_append_conn_error(conn, "could not open file \"%s\": %s",
- filename, strerror_r(errno, sebuf, sizeof(sebuf)));
+ filename, strerror_r(errno, sebuf, sizeof(sebuf)));
return InvalidOid;
}
@@ -723,8 +723,8 @@ lo_import_internal(PGconn *conn, const char *filename, Oid oid)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not read from file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return InvalidOid;
}
@@ -778,8 +778,8 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not open file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return -1;
}
@@ -799,8 +799,8 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
/* deliberately overwrite any error from lo_close */
pqClearConnErrorState(conn);
libpq_append_conn_error(conn, "could not write to file \"%s\": %s",
- filename,
- strerror_r(save_errno, sebuf, sizeof(sebuf)));
+ filename,
+ strerror_r(save_errno, sebuf, sizeof(sebuf)));
return -1;
}
}
@@ -822,7 +822,7 @@ lo_export(PGconn *conn, Oid lobjId, const char *filename)
if (close(fd) != 0 && result >= 0)
{
libpq_append_conn_error(conn, "could not write to file \"%s\": %s",
- filename, strerror_r(errno, sebuf, sizeof(sebuf)));
+ filename, strerror_r(errno, sebuf, sizeof(sebuf)));
result = -1;
}
@@ -954,56 +954,56 @@ lo_initialize(PGconn *conn)
if (lobjfuncs->fn_lo_open == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_open");
+ "lo_open");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_close == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_close");
+ "lo_close");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_creat == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_creat");
+ "lo_creat");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_unlink == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_unlink");
+ "lo_unlink");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_lseek == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_lseek");
+ "lo_lseek");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_tell == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lo_tell");
+ "lo_tell");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_read == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "loread");
+ "loread");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_write == 0)
{
libpq_append_conn_error(conn, "cannot determine OID of function %s",
- "lowrite");
+ "lowrite");
free(lobjfuncs);
return -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 4159610f6c..fadc46817b 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -749,8 +749,8 @@ retry4:
*/
definitelyEOF:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
@@ -1067,7 +1067,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
char sebuf[PG_STRERROR_R_BUFLEN];
libpq_append_conn_error(conn, "%s() failed: %s", "select",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
}
return result;
@@ -1280,7 +1280,7 @@ libpq_ngettext(const char *msgid, const char *msgid_plural, unsigned long n)
* newline.
*/
void
-libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...)
+libpq_append_error(PQExpBuffer errorMessage, const char *fmt,...)
{
int save_errno = errno;
bool done;
@@ -1309,7 +1309,7 @@ libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...)
* format should not end with a newline.
*/
void
-libpq_append_conn_error(PGconn *conn, const char *fmt, ...)
+libpq_append_conn_error(PGconn *conn, const char *fmt,...)
{
int save_errno = errno;
bool done;
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 364bad2b88..7a8e9c9962 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -466,7 +466,7 @@ static void
handleSyncLoss(PGconn *conn, char id, int msgLength)
{
libpq_append_conn_error(conn, "lost synchronization with server: got message type \"%c\", length %d",
- id, msgLength);
+ id, msgLength);
/* build an error result holding the error message */
pqSaveErrorResult(conn);
conn->asyncStatus = PGASYNC_READY; /* drop out of PQgetResult wait loop */
diff --git a/src/interfaces/libpq/fe-secure-common.c b/src/interfaces/libpq/fe-secure-common.c
index 7e4246c51f..6377e800dd 100644
--- a/src/interfaces/libpq/fe-secure-common.c
+++ b/src/interfaces/libpq/fe-secure-common.c
@@ -226,7 +226,7 @@ pq_verify_peer_name_matches_certificate_ip(PGconn *conn,
* wrong given the subject matter.
*/
libpq_append_conn_error(conn, "certificate contains IP address with invalid length %zu",
- iplen);
+ iplen);
return -1;
}
@@ -235,7 +235,7 @@ pq_verify_peer_name_matches_certificate_ip(PGconn *conn,
if (!addrstr)
{
libpq_append_conn_error(conn, "could not convert certificate's IP address to string: %s",
- strerror_r(errno, sebuf, sizeof(sebuf)));
+ strerror_r(errno, sebuf, sizeof(sebuf)));
return -1;
}
@@ -292,7 +292,7 @@ pq_verify_peer_name_matches_certificate(PGconn *conn)
else if (names_examined == 1)
{
libpq_append_conn_error(conn, "server certificate for \"%s\" does not match host name \"%s\"",
- first_name, host);
+ first_name, host);
}
else
{
diff --git a/src/interfaces/libpq/fe-secure-gssapi.c b/src/interfaces/libpq/fe-secure-gssapi.c
index 0ce92dbf43..af95dfa322 100644
--- a/src/interfaces/libpq/fe-secure-gssapi.c
+++ b/src/interfaces/libpq/fe-secure-gssapi.c
@@ -213,8 +213,8 @@ pg_GSS_write(PGconn *conn, const void *ptr, size_t len)
if (output.length > PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "client tried to send oversize GSSAPI packet (%zu > %zu)",
- (size_t) output.length,
- PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32));
+ (size_t) output.length,
+ PQ_GSS_SEND_BUFFER_SIZE - sizeof(uint32));
errno = EIO; /* for lack of a better idea */
goto cleanup;
}
@@ -349,8 +349,8 @@ pg_GSS_read(PGconn *conn, void *ptr, size_t len)
if (input.length > PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "oversize GSSAPI packet sent by the server (%zu > %zu)",
- (size_t) input.length,
- PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
+ (size_t) input.length,
+ PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
errno = EIO; /* for lack of a better idea */
return -1;
}
@@ -588,8 +588,8 @@ pqsecure_open_gss(PGconn *conn)
if (input.length > PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32))
{
libpq_append_conn_error(conn, "oversize GSSAPI packet sent by the server (%zu > %zu)",
- (size_t) input.length,
- PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
+ (size_t) input.length,
+ PQ_GSS_RECV_BUFFER_SIZE - sizeof(uint32));
return PGRES_POLLING_FAILED;
}
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index bad85359b6..4eb212d15c 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -213,12 +213,12 @@ rloop:
if (result_errno == EPIPE ||
result_errno == ECONNRESET)
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
else
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
}
else
{
@@ -313,12 +313,12 @@ pgtls_write(PGconn *conn, const void *ptr, size_t len)
result_errno = SOCK_ERRNO;
if (result_errno == EPIPE || result_errno == ECONNRESET)
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
else
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
}
else
{
@@ -410,7 +410,7 @@ pgtls_get_peer_certificate_hash(PGconn *conn, size_t *len)
if (algo_type == NULL)
{
libpq_append_conn_error(conn, "could not find digest for NID %s",
- OBJ_nid2sn(algo_nid));
+ OBJ_nid2sn(algo_nid));
return NULL;
}
break;
@@ -962,7 +962,7 @@ initialize_SSL(PGconn *conn)
if (ssl_min_ver == -1)
{
libpq_append_conn_error(conn, "invalid value \"%s\" for minimum SSL protocol version",
- conn->ssl_min_protocol_version);
+ conn->ssl_min_protocol_version);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -988,7 +988,7 @@ initialize_SSL(PGconn *conn)
if (ssl_max_ver == -1)
{
libpq_append_conn_error(conn, "invalid value \"%s\" for maximum SSL protocol version",
- conn->ssl_max_protocol_version);
+ conn->ssl_max_protocol_version);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1032,7 +1032,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read root certificate file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
SSL_CTX_free(SSL_context);
return -1;
@@ -1084,10 +1084,10 @@ initialize_SSL(PGconn *conn)
*/
if (fnbuf[0] == '\0')
libpq_append_conn_error(conn, "could not get home directory to locate root certificate file\n"
- "Either provide the file or change sslmode to disable server certificate verification.");
+ "Either provide the file or change sslmode to disable server certificate verification.");
else
libpq_append_conn_error(conn, "root certificate file \"%s\" does not exist\n"
- "Either provide the file or change sslmode to disable server certificate verification.", fnbuf);
+ "Either provide the file or change sslmode to disable server certificate verification.", fnbuf);
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1117,7 +1117,7 @@ initialize_SSL(PGconn *conn)
if (errno != ENOENT && errno != ENOTDIR)
{
libpq_append_conn_error(conn, "could not open certificate file \"%s\": %s",
- fnbuf, strerror_r(errno, sebuf, sizeof(sebuf)));
+ fnbuf, strerror_r(errno, sebuf, sizeof(sebuf)));
SSL_CTX_free(SSL_context);
return -1;
}
@@ -1135,7 +1135,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read certificate file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
SSL_CTX_free(SSL_context);
return -1;
@@ -1234,7 +1234,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not load SSL engine \"%s\": %s",
- engine_str, err);
+ engine_str, err);
SSLerrfree(err);
free(engine_str);
return -1;
@@ -1245,7 +1245,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not initialize SSL engine \"%s\": %s",
- engine_str, err);
+ engine_str, err);
SSLerrfree(err);
ENGINE_free(conn->engine);
conn->engine = NULL;
@@ -1260,7 +1260,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not read private SSL key \"%s\" from engine \"%s\": %s",
- engine_colon, engine_str, err);
+ engine_colon, engine_str, err);
SSLerrfree(err);
ENGINE_finish(conn->engine);
ENGINE_free(conn->engine);
@@ -1273,7 +1273,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "could not load private SSL key \"%s\" from engine \"%s\": %s",
- engine_colon, engine_str, err);
+ engine_colon, engine_str, err);
SSLerrfree(err);
ENGINE_finish(conn->engine);
ENGINE_free(conn->engine);
@@ -1310,10 +1310,10 @@ initialize_SSL(PGconn *conn)
{
if (errno == ENOENT)
libpq_append_conn_error(conn, "certificate present, but not private key file \"%s\"",
- fnbuf);
+ fnbuf);
else
libpq_append_conn_error(conn, "could not stat private key file \"%s\": %m",
- fnbuf);
+ fnbuf);
return -1;
}
@@ -1321,7 +1321,7 @@ initialize_SSL(PGconn *conn)
if (!S_ISREG(buf.st_mode))
{
libpq_append_conn_error(conn, "private key file \"%s\" is not a regular file",
- fnbuf);
+ fnbuf);
return -1;
}
@@ -1378,7 +1378,7 @@ initialize_SSL(PGconn *conn)
if (SSL_use_PrivateKey_file(conn->ssl, fnbuf, SSL_FILETYPE_ASN1) != 1)
{
libpq_append_conn_error(conn, "could not load private key file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
return -1;
}
@@ -1394,7 +1394,7 @@ initialize_SSL(PGconn *conn)
char *err = SSLerrmessage(ERR_get_error());
libpq_append_conn_error(conn, "certificate does not match private key file \"%s\": %s",
- fnbuf, err);
+ fnbuf, err);
SSLerrfree(err);
return -1;
}
@@ -1447,7 +1447,7 @@ open_client_SSL(PGconn *conn)
if (r == -1)
libpq_append_conn_error(conn, "SSL SYSCALL error: %s",
- SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
else
libpq_append_conn_error(conn, "SSL SYSCALL error: EOF detected");
pgtls_close(conn);
@@ -1489,12 +1489,12 @@ open_client_SSL(PGconn *conn)
case SSL_R_VERSION_TOO_LOW:
#endif
libpq_append_conn_error(conn, "This may indicate that the server does not support any SSL protocol version between %s and %s.",
- conn->ssl_min_protocol_version ?
- conn->ssl_min_protocol_version :
- MIN_OPENSSL_TLS_VERSION,
- conn->ssl_max_protocol_version ?
- conn->ssl_max_protocol_version :
- MAX_OPENSSL_TLS_VERSION);
+ conn->ssl_min_protocol_version ?
+ conn->ssl_min_protocol_version :
+ MIN_OPENSSL_TLS_VERSION,
+ conn->ssl_max_protocol_version ?
+ conn->ssl_max_protocol_version :
+ MAX_OPENSSL_TLS_VERSION);
break;
default:
break;
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index e74d3ccf69..215c9a74ed 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -255,14 +255,14 @@ pqsecure_raw_read(PGconn *conn, void *ptr, size_t len)
case EPIPE:
case ECONNRESET:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
- "\tThis probably means the server terminated abnormally\n"
- "\tbefore or while processing the request.");
+ "\tThis probably means the server terminated abnormally\n"
+ "\tbefore or while processing the request.");
break;
default:
libpq_append_conn_error(conn, "could not receive data from server: %s",
- SOCK_STRERROR(result_errno,
- sebuf, sizeof(sebuf)));
+ SOCK_STRERROR(result_errno,
+ sebuf, sizeof(sebuf)));
break;
}
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 512762f999..b4bb6db5a7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -888,8 +888,8 @@ extern char *libpq_ngettext(const char *msgid, const char *msgid_plural, unsigne
*/
#undef _
-extern void libpq_append_error(PQExpBuffer errorMessage, const char *fmt, ...) pg_attribute_printf(2, 3);
-extern void libpq_append_conn_error(PGconn *conn, const char *fmt, ...) pg_attribute_printf(2, 3);
+extern void libpq_append_error(PQExpBuffer errorMessage, const char *fmt,...) pg_attribute_printf(2, 3);
+extern void libpq_append_conn_error(PGconn *conn, const char *fmt,...) pg_attribute_printf(2, 3);
/*
* These macros are needed to let error-handling code be portable between
--
2.34.1
[application/octet-stream] v9-0002-Add-non-blocking-version-of-PQcancel.patch (55.7K, ../../DBBPR83MB05071BB8A8838830CABAFFE1F7129@DBBPR83MB0507.EURPRD83.prod.outlook.com/3-v9-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 4103ac630049bd0c9ea25105c2220273901077d7 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Wed, 12 Jan 2022 09:52:05 +0100
Subject: [PATCH v9 2/2] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
3. Use these two new cancellation APIs everywhere in the codebase where
signal-safety is not a necessity.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests. The postgres_fdw
cancellation code has been modified to make use of this.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
contrib/dblink/dblink.c | 22 +-
contrib/postgres_fdw/connection.c | 93 ++++-
.../postgres_fdw/expected/postgres_fdw.out | 15 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +
doc/src/sgml/libpq.sgml | 279 +++++++++++--
src/fe_utils/connect_utils.c | 10 +-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 373 ++++++++++++++++--
src/interfaces/libpq/fe-misc.c | 15 +-
src/interfaces/libpq/fe-secure-openssl.c | 2 +-
src/interfaces/libpq/fe-secure.c | 6 +
src/interfaces/libpq/libpq-fe.h | 25 +-
src/interfaces/libpq/libpq-int.h | 9 +
src/test/isolation/isolationtester.c | 29 +-
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++-
15 files changed, 1048 insertions(+), 109 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 04095a8f0e..05058dc66b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1378,22 +1378,24 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
-
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ cancelConn = PQcancelSend(conn);
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ {
+ msg = "OK";
+ }
+ PQcancelFinish(cancelConn);
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index f0c45b00db..ce5f908bb1 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -1264,35 +1264,98 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
PGresult *result = NULL;
- TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ return false;
+ }
+
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ }
+ PG_CATCH();
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PQcancelFinish(cancel_conn);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PQcancelFinish(cancel_conn);
+ return failed;
}
+ PQcancelFinish(cancel_conn);
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 2ab3f1efaa..1036ebc336 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2688,6 +2688,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 51560429e0..0923f93803 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -326,6 +326,7 @@ DELETE FROM loct_empty;
ANALYZE ft_empty;
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM ft_empty ORDER BY c1;
+
-- ===================================================================
-- WHERE with remotely-executable conditions
-- ===================================================================
@@ -713,6 +714,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- cleanup
DROP OWNED BY regress_view_owner;
DROP ROLE regress_view_owner;
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index f9558dec3b..2e21ea6ee7 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -4909,7 +4909,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -5627,13 +5627,220 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command.
+<synopsis>
+PGcancelConn *PQcancelSend(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ This request is made over a connection that uses the same connection
+ options as the the original <structname>PGconn</structname>. So when the
+ original connection is encrypted (using TLS or GSS), the connection for
+ the cancel request connection is encrypted in the same. Any connection
+ options that only make sense for authentication or after authentication
+ are ignored though, because cancellation requests do not require
+ authentication.
+ </para>
+
+ <para>
+ This function returns a <structname>PGcancelConn</structname>
+ object. By using
+ <xref linkend="libpq-PQcancelStatus"/>
+ it can be checked if there was any error when sending the cancellation
+ request. If <xref linkend="libpq-PQcancelStatus"/>
+ returns for <symbol>CONNECTION_OK</symbol> the request was
+ successfully sent, but if it returns <symbol>CONNECTION_BAD</symbol>
+ an error occured. If an error occured the error message can be retrieved using
+ <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being cancelled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelSend</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQcancelSend"/> that can be used
+ in a non-blocking manner.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>,
+ but it won't instantly start sending a cancel request over this
+ connection like <xref linkend="libpq-PQcancelSend"/>.
+ <xref linkend="libpq-PQcancelStatus"/> should be called on the return
+ value to check if the <structname> PGcancelConn </structname> was
+ created successfully.
+ The <structname>PGcancelConn</structname> object is an opaque structure
+ that is not meant to be accessed directly by the application.
+ This <structname>PGcancelConn</structname> object can be used to cancel
+ the query that's running on the original connection in a thread-safe and
+ non-blocking way.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually cancelled).
+ While a <symbol>CONNECTION_OK</symbol> result for
+ <structname>PGconn</structname> means thatqueries can be sent over the
+ connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -5675,14 +5882,30 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of
+ <xref linkend="libpq-PQcancelSend"/>, but one that can be used safely
+ from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead.
+ <xref linkend="libpq-PQcancel"/> can be safely invoked from a signal
+ handler, if the <parameter>errbuf</parameter> is a local variable in the
+ signal handler. The <structname>PGcancel</structname> object is read-only
+ as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
+ also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -5690,21 +5913,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. When calling this function a
+ connection is made to the postgres host using the same port. The only
+ connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -5716,13 +5940,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -8871,7 +9104,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 1cc97b72f7..0f5e84ad71 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,11 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
-
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ PQcancelFinish(PQcancelSend(conn));
}
PQfinish(conn);
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..f56e8c185c 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,11 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQcancelSend 187
+PQcancelConn 188
+PQcancelPoll 189
+PQcancelStatus 190
+PQcancelSocket 191
+PQcancelErrorMessage 192
+PQcancelReset 193
+PQcancelFinish 194
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d0c3b21fb9..364fd4f032 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -376,6 +376,7 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
@@ -599,8 +600,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQresetStart invocations. Otherwise they don't know the secret token of
+ * the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -731,6 +741,68 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a request on the given connection. This requires
+ * polling the returned PGconn to actually complete the cancellation of the
+ * request.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancelation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used.
+ */
+ memcpy(&cancelConn->raddr, &conn->raddr, sizeof(SockAddr));
+ cancelConn->whichhost = conn->whichhost;
+ conn->try_next_host = false;
+ conn->try_next_addr = false;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -906,6 +978,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2030,10 +2141,17 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host,
+ * which is determined in PQcancelConn. So leave these settings alone for
+ * cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2082,6 +2200,15 @@ connectDBComplete(PGconn *conn)
if (conn == NULL || conn->status == CONNECTION_BAD)
return 0;
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return 0;
+ }
+ }
+
/*
* Set up a time limit, if connect_timeout isn't zero.
*/
@@ -2222,8 +2349,8 @@ PQconnectPoll(PGconn *conn)
switch (conn->status)
{
/*
- * We really shouldn't have been polled in these two cases, but we
- * can handle it.
+ * We really shouldn't have been polled in these three cases, but
+ * we can handle it.
*/
case CONNECTION_BAD:
return PGRES_POLLING_FAILED;
@@ -2240,6 +2367,34 @@ PQconnectPoll(PGconn *conn)
/* Load waiting data */
int n = pqReadData(conn);
+#ifndef WIN32
+ if (n == -2 && conn->cancelRequest)
+#else
+
+ /*
+ * Windows is a bit special in its EOF behaviour for TCP.
+ * Sometimes it will error with an ECONNRESET when there is a
+ * clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the
+ * cancellation anyway, so even if this is not always correct
+ * we do the same here.
+ */
+ if (n < 0 && conn->cancelRequest)
+#endif
+ {
+ /*
+ * This is the expected end state for cancel connections.
+ * They are closed once the cancel is processed by the
+ * server.
+ */
+ conn->status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+ }
if (n < 0)
goto error_return;
if (n == 0)
@@ -2249,6 +2404,7 @@ PQconnectPoll(PGconn *conn)
}
/* These are writing states, so we just proceed. */
+ case CONNECTION_STARTING:
case CONNECTION_STARTED:
case CONNECTION_MADE:
break;
@@ -2272,6 +2428,14 @@ keep_going: /* We will come back to here until there is
/* Time to advance to next address, or next host if no more addresses? */
if (conn->try_next_addr)
{
+ /*
+ * Cancel requests never have more addresses to try. They should only
+ * try a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
if (conn->addr_cur && conn->addr_cur->ai_next)
{
conn->addr_cur = conn->addr_cur->ai_next;
@@ -2291,6 +2455,15 @@ keep_going: /* We will come back to here until there is
int ret;
char portstr[MAXPGPATH];
+ /*
+ * Cancel requests never have more hosts to try. They should only try
+ * a single one.
+ */
+ if (conn->cancelRequest)
+ {
+ goto error_return;
+ }
+
if (conn->whichhost + 1 < conn->nconnhost)
conn->whichhost++;
else
@@ -2466,19 +2639,27 @@ keep_going: /* We will come back to here until there is
char host_addr[NI_MAXHOST];
/*
- * Advance to next possible host, if we've tried all of
- * the addresses for the current host.
+ * Cancel requests don't use addr_cur at all. They have
+ * their raddr field already filled in during
+ * initialization in PQcancelConn.
*/
- if (addr_cur == NULL)
+ if (!conn->cancelRequest)
{
- conn->try_next_host = true;
- goto keep_going;
- }
+ /*
+ * Advance to next possible host, if we've tried all
+ * of the addresses for the current host.
+ */
+ if (addr_cur == NULL)
+ {
+ conn->try_next_host = true;
+ goto keep_going;
+ }
- /* Remember current address for possible use later */
- memcpy(&conn->raddr.addr, addr_cur->ai_addr,
- addr_cur->ai_addrlen);
- conn->raddr.salen = addr_cur->ai_addrlen;
+ /* Remember current address for possible use later */
+ memcpy(&conn->raddr.addr, addr_cur->ai_addr,
+ addr_cur->ai_addrlen);
+ conn->raddr.salen = addr_cur->ai_addrlen;
+ }
/*
* Set connip, too. Note we purposely ignore strdup
@@ -2494,7 +2675,7 @@ keep_going: /* We will come back to here until there is
conn->connip = strdup(host_addr);
/* Try to create the socket */
- conn->sock = socket(addr_cur->ai_family, SOCK_STREAM, 0);
+ conn->sock = socket(conn->raddr.addr.ss_family, SOCK_STREAM, 0);
if (conn->sock == PGINVALID_SOCKET)
{
int errorno = SOCK_ERRNO;
@@ -2504,12 +2685,18 @@ keep_going: /* We will come back to here until there is
* addresses to try; this reduces useless chatter in
* cases where the address list includes both IPv4 and
* IPv6 but kernel only accepts one family.
+ *
+ * Cancel requests never have more addresses to try.
+ * They should only try a single one.
*/
- if (addr_cur->ai_next != NULL ||
- conn->whichhost + 1 < conn->nconnhost)
+ if (!conn->cancelRequest)
{
- conn->try_next_addr = true;
- goto keep_going;
+ if (addr_cur->ai_next != NULL ||
+ conn->whichhost + 1 < conn->nconnhost)
+ {
+ conn->try_next_addr = true;
+ goto keep_going;
+ }
}
emitHostIdentityInfo(conn, host_addr);
libpq_append_conn_error(conn, "could not create socket: %s",
@@ -2531,7 +2718,7 @@ keep_going: /* We will come back to here until there is
* TCP sockets, nonblock mode, close-on-exec. Try the
* next address if any of this fails.
*/
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
if (!connectNoDelay(conn))
{
@@ -2558,7 +2745,7 @@ keep_going: /* We will come back to here until there is
}
#endif /* F_SETFD */
- if (addr_cur->ai_family != AF_UNIX)
+ if (conn->raddr.addr.ss_family != AF_UNIX)
{
#ifndef WIN32
int on = 1;
@@ -2650,8 +2837,9 @@ keep_going: /* We will come back to here until there is
* Start/make connection. This should not block, since we
* are in nonblock mode. If it does, well, too bad.
*/
- if (connect(conn->sock, addr_cur->ai_addr,
- addr_cur->ai_addrlen) < 0)
+ if (connect(conn->sock,
+ (struct sockaddr *) &conn->raddr.addr,
+ conn->raddr.salen) < 0)
{
if (SOCK_ERRNO == EINPROGRESS ||
#ifdef WIN32
@@ -2690,6 +2878,16 @@ keep_going: /* We will come back to here until there is
}
}
+ case CONNECTION_STARTING:
+ {
+ if (!connectDBStart(conn))
+ {
+ goto error_return;
+ }
+ conn->status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+
case CONNECTION_STARTED:
{
socklen_t optlen = sizeof(optval);
@@ -2891,6 +3089,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancelation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4063,6 +4284,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4531,6 +4761,96 @@ cancel_errReturn:
return false;
}
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+PGcancelConn *
+PQcancelSend(PGconn *conn)
+{
+ PGcancelConn *cancelConn = PQcancelConn(conn);
+
+ if (cancelConn && cancelConn->conn.status != CONNECTION_BAD)
+ (void) connectDBComplete(&cancelConn->conn);
+
+ return cancelConn;
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ return PQconnectPoll((PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
@@ -4589,6 +4909,7 @@ PQrequestCancel(PGconn *conn)
}
+
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index fadc46817b..40a7b9ab71 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -556,8 +556,11 @@ pqPutMsgEnd(PGconn *conn)
* Possible return values:
* 1: successfully loaded at least one more byte
* 0: no data is presently available, but no error detected
- * -1: error detected (including EOF = connection closure);
+ * -1: error detected (excluding EOF = connection closure);
* conn->errorMessage set
+ * -2: EOF detected, connection is closed
+ * conn->errorMessage set
+ *
* NOTE: callers must not assume that pointers or indexes into conn->inBuffer
* remain valid across this call!
* ----------
@@ -639,7 +642,7 @@ retry3:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -734,7 +737,7 @@ retry4:
default:
/* pqsecure_read set the error message for us */
- return -1;
+ return nread;
}
}
if (nread > 0)
@@ -751,13 +754,17 @@ definitelyEOF:
libpq_append_conn_error(conn, "server closed the connection unexpectedly\n"
"\tThis probably means the server terminated abnormally\n"
"\tbefore or while processing the request.");
+ /* Do *not* drop any already-read data; caller still wants it */
+ pqDropConnection(conn, false);
+ conn->status = CONNECTION_BAD; /* No more connection to backend */
+ return -2;
/* Come here if lower-level code already set a suitable errorMessage */
definitelyFailed:
/* Do *not* drop any already-read data; caller still wants it */
pqDropConnection(conn, false);
conn->status = CONNECTION_BAD; /* No more connection to backend */
- return -1;
+ return nread < 0 ? nread : -1;
}
/*
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 4eb212d15c..1b83946aee 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -248,7 +248,7 @@ rloop:
*/
libpq_append_conn_error(conn, "SSL connection has been closed unexpectedly");
result_errno = ECONNRESET;
- n = -1;
+ n = -2;
break;
default:
libpq_append_conn_error(conn, "unrecognized SSL error code: %d", err);
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 215c9a74ed..2cbfa501e2 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -199,6 +199,12 @@ pqsecure_close(PGconn *conn)
* On failure, this function is responsible for appending a suitable message
* to conn->errorMessage. The caller must still inspect errno, but only
* to determine whether to continue/retry after error.
+ *
+ * Returns -1 in case of failures, except in the case of where a failure means
+ * that there was a clean connection closure, in those cases -2 is returned.
+ * Currently only the TLS implementation of pqsecure_read ever returns -2. For
+ * the other implementations a clean connection closure is detected in
+ * pqReadData instead.
*/
ssize_t
pqsecure_read(PGconn *conn, void *ptr, size_t len)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index b7df3224c0..697e0ed85d 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,28 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* issue a cancel request */
+extern PGcancelConn * PQcancelSend(PGconn *conn);
+/* non-blocking version of PQrequestSend */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b4bb6db5a7..6e117f8b86 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -397,6 +397,10 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -592,6 +596,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153..3781f7982b 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelSend(conn);
- if (cancel != NULL)
+ if (PQcancelStatus(cancel_conn) == CONNECTION_OK)
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index a37e4e2500..4018c61a3b 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got cancelled.
+ *
+ * This is a function wrapped in a macrco to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_cancelled(conn) confirm_query_cancelled_impl(__LINE__, conn)
+static void
+confirm_query_cancelled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_cancelled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_cancelled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelSend(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_cancelled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1681,6 +1939,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1782,7 +2041,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 01:59 vignesh C <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: vignesh C @ 2024-01-26 01:59 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Wed, 20 Dec 2023 at 19:17, Jelte Fennema-Nio <[email protected]> wrote:
>
> On Thu, 14 Dec 2023 at 13:57, Jelte Fennema-Nio <[email protected]> wrote:
> > I changed all the places that were not adhering to those spellings.
>
> It seems I forgot a /g on my sed command to do this so it turned out I
> missed one that caused the test to fail to compile... Attached is a
> fixed version.
>
> I also updated the patchset to use the EOF detection provided by
> 0a5c46a7a488f2f4260a90843bb9de6c584c7f4e instead of introducing a new
> way of EOF detection using a -2 return value.
CFBot shows that the patch does not apply anymore as in [1]:
patching file doc/src/sgml/libpq.sgml
...
patching file src/interfaces/libpq/exports.txt
Hunk #1 FAILED at 191.
1 out of 1 hunk FAILED -- saving rejects to file
src/interfaces/libpq/exports.txt.rej
patching file src/interfaces/libpq/fe-connect.c
Please post an updated version for the same.
[1] - http://cfbot.cputube.org/patch_46_3511.log
Regards,
Vignesh
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 10:44 Jelte Fennema-Nio <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 10:44 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 02:59, vignesh C <[email protected]> wrote:
> Please post an updated version for the same.
Done.
Attachments:
[application/x-patch] v25-0002-Add-non-blocking-version-of-PQcancel.patch (43.8K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/2-v25-0002-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 5a94d610a4fe138365e2e88c5cec72eba53ed036 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:04 +0100
Subject: [PATCH v25 2/3] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++-
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-connect.c | 451 +++++++++++++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 9 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++-
6 files changed, 987 insertions(+), 51 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..f8e3b5953f0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -394,8 +394,10 @@ static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
+static bool copyPGconn(PGconn *srcConn, PGconn *dstConn);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
+static void release_conn_hosts(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -623,8 +625,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -755,6 +766,113 @@ PQping(const char *conninfo)
return ret;
}
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = makeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!copyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!connectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
/*
* PQconnectStartParams
*
@@ -930,6 +1048,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+static bool
+copyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2361,10 +2518,18 @@ connectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2506,7 +2671,10 @@ connectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2631,13 +2799,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3279,6 +3451,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4028,8 +4223,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4397,19 +4598,8 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ release_conn_addrinfo(conn);
+ release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4528,6 +4718,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+static void
+release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
@@ -4535,6 +4750,15 @@ release_conn_addrinfo(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4588,7 +4812,13 @@ closePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
@@ -5003,6 +5233,179 @@ cancel_errReturn:
return false;
}
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!connectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return connectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using connectDBstart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!connectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ closePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
/*
* PQrequestCancel: old, not thread-safe function for requesting query cancel
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..ea99ff4efa5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/x-patch] v25-0003-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/3-v25-0003-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From 2ee4f68919d60bd94f7ecfc23a662a22efaf0e62 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v25 3/3] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index d83f6ae8cbc..df0b88e70c7 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 90c8fa4b705..115c3c117a1 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/x-patch] v25-0001-Fix-spelling-of-canceled-cancellation.patch (3.1K, ../../CAGECzQTUMO5eE3m2bBeiitj1ObQ0JbZ-NEF0RqLSnXp2vwrttw@mail.gmail.com/4-v25-0001-Fix-spelling-of-canceled-cancellation.patch)
download | inline diff:
From 2c43b57abe766a0b817ee85597a326f3ece5ed41 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:31:18 +0100
Subject: [PATCH v25 1/3] Fix spelling of canceled/cancellation
This fixes places where words derived from cancel were not using their
common en-US spelling.
---
doc/src/sgml/event-trigger.sgml | 2 +-
doc/src/sgml/libpq.sgml | 2 +-
src/backend/storage/lmgr/proc.c | 2 +-
src/test/recovery/t/001_stream_rep.pl | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/event-trigger.sgml b/doc/src/sgml/event-trigger.sgml
index 234b4ffd024..a76bd844257 100644
--- a/doc/src/sgml/event-trigger.sgml
+++ b/doc/src/sgml/event-trigger.sgml
@@ -50,7 +50,7 @@
writing anything to the database when running on a standby.
Also, it's recommended to avoid long-running queries in
<literal>login</literal> event triggers. Notes that, for instance,
- cancelling connection in <application>psql</application> wouldn't cancel
+ canceling connection in <application>psql</application> wouldn't cancel
the in-progress <literal>login</literal> trigger.
</para>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 173ab779a08..d0d5aefadc0 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -7625,7 +7625,7 @@ defaultNoticeProcessor(void *arg, const char *message)
is called. It is the ideal time to initialize any
<literal>instanceData</literal> an event procedure may need. Only one
register event will be fired per event handler per connection. If the
- event procedure fails (returns zero), the registration is cancelled.
+ event procedure fails (returns zero), the registration is canceled.
<synopsis>
typedef struct
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4ad96beb87a..e5977548fe2 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -1353,7 +1353,7 @@ ProcSleep(LOCALLOCK *locallock, LockMethod lockMethodTable)
* coding means that there is a tiny chance that the process
* terminates its current transaction and starts a different one
* before we have a change to send the signal; the worst possible
- * consequence is that a for-wraparound vacuum is cancelled. But
+ * consequence is that a for-wraparound vacuum is canceled. But
* that could happen in any case unless we were to do kill() with
* the lock held, which is much more undesirable.
*/
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
index cb988f4d10c..5311ade509b 100644
--- a/src/test/recovery/t/001_stream_rep.pl
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -601,7 +601,7 @@ is( $node_primary->poll_query_until(
ok( pump_until(
$sigchld_bb, $sigchld_bb_timeout,
\$sigchld_bb_stderr, qr/backup is not in progress/),
- 'base backup cleanly cancelled');
+ 'base backup cleanly canceled');
$sigchld_bb->finish();
done_testing();
base-commit: b199eb89c67d737ced55721590f7fc8ff585e837
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 12:11 Alvaro Herrera <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Alvaro Herrera @ 2024-01-26 12:11 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
Pushed 0001.
I wonder, would it make sense to put all these new functions in a
separate file fe-cancel.c?
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"World domination is proceeding according to plan" (Andrew Morton)
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 16:52 Jelte Fennema-Nio <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 2 replies; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 16:52 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 13:11, Alvaro Herrera <[email protected]> wrote:
> I wonder, would it make sense to put all these new functions in a
> separate file fe-cancel.c?
Okay I tried doing that. I think the end result is indeed quite nice,
having all the cancellation related functions together in a file. But
it did require making a bunch of static functions in fe-connect
extern, and adding them to libpq-int.h. On one hand that seems fine to
me, on the other maybe that indicates that this cancellation logic
makes sense to be in the same file as the other connection functions
(in a sense, connecting is all that a cancel request does).
Attachments:
[application/octet-stream] v26-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/2-v26-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From 0a8de21ec8728a474d89d19880d44efae39038ef Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v26 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v26-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/3-v26-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 63fb41b9aa654a6450a4c2f08d8bf204a5916b08 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v26 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 39 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5357b0a9d22..0622fe32253 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66b77e75e18..a0da7356584 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v26-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/4-v26-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From c727e1ccab265c49f7737ba083dd0bf1aa55471e Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v26 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 0622fe32253..8dbc9d2cc57 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4563,15 +4558,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4614,7 +4609,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4628,9 +4623,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4661,9 +4656,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index a0da7356584..b1e1bd6331f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v26-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (26.5K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/5-v26-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From b3a3e5e659b68f2a9abf1d9af8733ee4888c5c60 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v26 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/fe-cancel.c | 386 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 405 ++----------------------------
src/interfaces/libpq/libpq-int.h | 2 +
src/interfaces/libpq/meson.build | 1 +
4 files changed, 407 insertions(+), 387 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..f1d836d0216
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,386 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!setKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5357b0a9d22 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2269,12 +2267,12 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (!setKeepalivesWin32(conn->sock, idle, interval))
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..66b77e75e18 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,6 +678,8 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: f7cf9494bad3aef1b2ba1cd84376a1e71797ac50
--
2.34.1
[application/octet-stream] v26-0004-Add-non-blocking-version-of-PQcancel.patch (44.1K, ../../CAGECzQR80k0OMCS+AC08r_rw1w3rvie-1TrZ2zTUB-mPs9uTTw@mail.gmail.com/6-v26-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 67f311ca416361668a6865d2d893f8c6d0cb6cd0 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v26 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 304 +++++++++++++++++-
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 ++++++++++++++-
7 files changed, 974 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index f1d836d0216..e9262359453 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -19,6 +19,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
@@ -55,36 +339,36 @@ PQgetCancel(PGconn *conn)
if (conn->pgtcp_user_timeout != NULL)
{
if (!pq_parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
goto fail;
}
if (conn->keepalives != NULL)
{
if (!pq_parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
+ &cancel->keepalives,
+ conn, "keepalives"))
goto fail;
}
if (conn->keepalives_idle != NULL)
{
if (!pq_parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
goto fail;
}
if (conn->keepalives_interval != NULL)
{
if (!pq_parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
goto fail;
}
if (conn->keepalives_count != NULL)
{
if (!pq_parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
goto fail;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8dbc9d2cc57..b63ac63d514 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4541,6 +4634,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4594,7 +4696,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b1e1bd6331f..94990292a04 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 17:19 Alvaro Herrera <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
1 sibling, 1 reply; 36+ messages in thread
From: Alvaro Herrera @ 2024-01-26 17:19 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 2024-Jan-26, Jelte Fennema-Nio wrote:
> Okay I tried doing that. I think the end result is indeed quite nice,
> having all the cancellation related functions together in a file. But
> it did require making a bunch of static functions in fe-connect
> extern, and adding them to libpq-int.h. On one hand that seems fine to
> me, on the other maybe that indicates that this cancellation logic
> makes sense to be in the same file as the other connection functions
> (in a sense, connecting is all that a cancel request does).
Yeah, I see that point of view as well. I like the end result; the
additional protos in libpq-int.h don't bother me. Does anybody else
wants to share their opinion on it? If none, then I'd consider going
ahead with this version.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"We’ve narrowed the problem down to the customer’s pants being in a situation
of vigorous combustion" (Robert Haas, Postgres expert extraordinaire)
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-26 23:14 Jelte Fennema-Nio <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-26 23:14 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 18:19, Alvaro Herrera <[email protected]> wrote:
> Yeah, I see that point of view as well. I like the end result; the
> additional protos in libpq-int.h don't bother me. Does anybody else
> wants to share their opinion on it? If none, then I'd consider going
> ahead with this version.
To be clear, I'm +1 on the new file structure (although if people feel
strongly against it, I don't care enough to make a big deal out of
it).
@Alvaro did you have any other comments on the contents of the patch btw?
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-28 03:15 vignesh C <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
1 sibling, 1 reply; 36+ messages in thread
From: vignesh C @ 2024-01-28 03:15 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Fri, 26 Jan 2024 at 22:22, Jelte Fennema-Nio <[email protected]> wrote:
>
> On Fri, 26 Jan 2024 at 13:11, Alvaro Herrera <[email protected]> wrote:
> > I wonder, would it make sense to put all these new functions in a
> > separate file fe-cancel.c?
>
>
> Okay I tried doing that. I think the end result is indeed quite nice,
> having all the cancellation related functions together in a file. But
> it did require making a bunch of static functions in fe-connect
> extern, and adding them to libpq-int.h. On one hand that seems fine to
> me, on the other maybe that indicates that this cancellation logic
> makes sense to be in the same file as the other connection functions
> (in a sense, connecting is all that a cancel request does).
CFBot shows that the patch has few compilation errors as in [1]:
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`handle_sigint':
[17:07:07.621] cancel.c:(.text+0x50): undefined reference to `PQcancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`SetCancelConn':
[17:07:07.621] cancel.c:(.text+0x10c): undefined reference to `PQfreeCancel'
[17:07:07.621] /usr/bin/ld: cancel.c:(.text+0x114): undefined
reference to `PQgetCancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
`ResetCancelConn':
[17:07:07.621] cancel.c:(.text+0x148): undefined reference to `PQfreeCancel'
[17:07:07.621] /usr/bin/ld:
../../../src/fe_utils/libpgfeutils.a(connect_utils.o): in function
`disconnectDatabase':
[17:07:07.621] connect_utils.c:(.text+0x2fc): undefined reference to
`PQcancelConn'
[17:07:07.621] /usr/bin/ld: connect_utils.c:(.text+0x307): undefined
reference to `PQcancelSend'
[17:07:07.621] /usr/bin/ld: connect_utils.c:(.text+0x30f): undefined
reference to `PQcancelFinish'
[17:07:07.623] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:07.626] collect2: error: ld returned 1 exit status
[17:07:07.626] make[3]: *** [Makefile:31: pg_amcheck] Error 1
[17:07:07.626] make[2]: *** [Makefile:45: all-pg_amcheck-recurse] Error 2
[17:07:07.626] make[2]: *** Waiting for unfinished jobs....
[17:07:08.126] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:08.130] collect2: error: ld returned 1 exit status
[17:07:08.131] make[3]: *** [Makefile:42: initdb] Error 1
[17:07:08.131] make[2]: *** [Makefile:45: all-initdb-recurse] Error 2
[17:07:08.492] /usr/bin/ld: ../../../src/interfaces/libpq/libpq.so:
undefined reference to `PQcancelPoll'
[17:07:08.495] collect2: error: ld returned 1 exit status
[17:07:08.496] make[3]: *** [Makefile:50: pg_basebackup] Error 1
[17:07:08.496] make[2]: *** [Makefile:45: all-pg_basebackup-recurse] Error 2
[17:07:09.060] /usr/bin/ld: parallel.o: in function `sigTermHandler':
[17:07:09.060] parallel.c:(.text+0x1aa): undefined reference to `PQcancel'
Please post an updated version for the same.
[1] - https://cirrus-ci.com/task/6210637211107328
Regards,
Vignesh
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-28 09:51 Jelte Fennema-Nio <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-28 09:51 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Sun, 28 Jan 2024 at 04:15, vignesh C <[email protected]> wrote:
> CFBot shows that the patch has few compilation errors as in [1]:
> [17:07:07.621] /usr/bin/ld:
> ../../../src/fe_utils/libpgfeutils.a(cancel.o): in function
> `handle_sigint':
> [17:07:07.621] cancel.c:(.text+0x50): undefined reference to `PQcancel'
I forgot to update ./configure based builds with the new file, only
meson was working. Also it seems I trimmed the header list fe-cancel.c
a bit too much for OSX, so I added unistd.h back.
Both of those are fixed now.
Attachments:
[application/octet-stream] v27-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/2-v27-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 134d698f01e430fa8f119a4804888e022b33a68e Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v27 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 39 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5357b0a9d22..0622fe32253 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,31 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66b77e75e18..a0da7356584 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v27-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/3-v27-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From c9e1b1d66efddf9f15a22daf450757c6d6561775 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v27 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v27-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/4-v27-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From 5851e24c40ee68a6da3837f27a6d0cab18322b94 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v27 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 0622fe32253..8dbc9d2cc57 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4563,15 +4558,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4614,7 +4609,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4628,9 +4623,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4661,9 +4656,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index a0da7356584..b1e1bd6331f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/octet-stream] v27-0004-Add-non-blocking-version-of-PQcancel.patch (44.1K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/5-v27-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From badc854b51348b7b36a873bc67886b3d247d6506 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v27 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 ++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 304 +++++++++++++++++-
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 ++++++++++++++-
7 files changed, 974 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index 9626f17a9cd..fd2a4f5b209 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
@@ -57,36 +341,36 @@ PQgetCancel(PGconn *conn)
if (conn->pgtcp_user_timeout != NULL)
{
if (!pq_parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
goto fail;
}
if (conn->keepalives != NULL)
{
if (!pq_parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
+ &cancel->keepalives,
+ conn, "keepalives"))
goto fail;
}
if (conn->keepalives_idle != NULL)
{
if (!pq_parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
goto fail;
}
if (conn->keepalives_interval != NULL)
{
if (!pq_parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
goto fail;
}
if (conn->keepalives_count != NULL)
{
if (!pq_parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
goto fail;
}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8dbc9d2cc57..b63ac63d514 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4541,6 +4634,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4594,7 +4696,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index b1e1bd6331f..94990292a04 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/octet-stream] v27-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (26.9K, ../../CAGECzQSszLdgdrTupMKOMV6+WPB0QQXm-U04k3xy_8jmWBM1tw@mail.gmail.com/6-v27-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From 98d8e783ebf4afa824bec70fd3ed266e2dba6908 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v27 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/fe-cancel.c | 388 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 405 ++----------------------------
src/interfaces/libpq/libpq-int.h | 2 +
src/interfaces/libpq/meson.build | 1 +
5 files changed, 410 insertions(+), 387 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fce17bc72a0..bfcc7cdde99 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -30,6 +30,7 @@ endif
OBJS = \
$(WIN32RES) \
fe-auth-scram.o \
+ fe-cancel.o \
fe-connect.o \
fe-exec.o \
fe-lobj.o \
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..9626f17a9cd
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,388 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <unistd.h>
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!setKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5357b0a9d22 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2269,12 +2267,12 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (!setKeepalivesWin32(conn->sock, idle, interval))
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..66b77e75e18 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,6 +678,8 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: a3a836fb5e51183eae624d43225279306c2285b8
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-28 12:39 Jelte Fennema-Nio <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-28 12:39 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Sun, 28 Jan 2024 at 10:51, Jelte Fennema-Nio <[email protected]> wrote:
> Both of those are fixed now.
Okay, there turned out to also be an issue on Windows with
setKeepalivesWin32 not being available in fe-cancel.c. That's fixed
now too (as well as some minor formatting issues).
Attachments:
[application/x-patch] v28-0002-libpq-Add-pq_release_conn_hosts-function.patch (2.5K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/2-v28-0002-libpq-Add-pq_release_conn_hosts-function.patch)
download | inline diff:
From 4efbb0c75341f4612f0c5b8d5d3fe3f8f9c3b43c Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v28 2/5] libpq: Add pq_release_conn_hosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 38 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 5d08b4904d3..bc1f6521650 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4395,19 +4395,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4526,6 +4514,30 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pq_release_conn_hosts
+ * - Free the host list in the PGconn.
+ */
+void
+pq_release_conn_hosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 48c10b474f5..4cbad2c2c83 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -680,6 +680,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pq_release_conn_hosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/x-patch] v28-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/3-v28-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From f1168ac4c3dd758a77be3ceb8c40bacb9aebef8c Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v28 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc1f6521650..aeb3adc0e31 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2323,14 +2318,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2393,14 +2388,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2750,7 +2745,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4227,7 +4222,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4279,11 +4274,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4376,7 +4371,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4562,15 +4557,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4613,7 +4608,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4627,9 +4622,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4660,9 +4655,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4cbad2c2c83..c1ff12dd396 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -681,6 +681,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
[application/x-patch] v28-0001-libpq-Move-cancellation-related-functions-to-fe-.patch (27.7K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/4-v28-0001-libpq-Move-cancellation-related-functions-to-fe-.patch)
download | inline diff:
From d5cdd1451ecd9160d285bdfe3cdcf6df452c5249 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 14:35:48 +0100
Subject: [PATCH v28 1/5] libpq: Move cancellation related functions to
fe-cancel.c
In follow up commits we'll add more functions related to cancellations
this groups those all together instead of grouping them with all the
other functions in fe-connect.c
---
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/fe-cancel.c | 387 ++++++++++++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 411 ++----------------------------
src/interfaces/libpq/libpq-int.h | 6 +
src/interfaces/libpq/meson.build | 1 +
5 files changed, 416 insertions(+), 390 deletions(-)
create mode 100644 src/interfaces/libpq/fe-cancel.c
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fce17bc72a0..bfcc7cdde99 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -30,6 +30,7 @@ endif
OBJS = \
$(WIN32RES) \
fe-auth-scram.o \
+ fe-cancel.o \
fe-connect.o \
fe-exec.o \
fe-lobj.o \
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
new file mode 100644
index 00000000000..ce28d39f3f5
--- /dev/null
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -0,0 +1,387 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-cancel.c
+ * functions related to setting up a connection to the backend
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-cancel.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <unistd.h>
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "port/pg_bswap.h"
+
+/*
+ * PQgetCancel: get a PGcancel structure corresponding to a connection.
+ *
+ * A copy is needed to be able to cancel a running query from a different
+ * thread. If the same structure is used all structure members would have
+ * to be individually locked (if the entire structure was locked, it would
+ * be impossible to cancel a synchronous query because the structure would
+ * have to stay locked for the duration of the query).
+ */
+PGcancel *
+PQgetCancel(PGconn *conn)
+{
+ PGcancel *cancel;
+
+ if (!conn)
+ return NULL;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ return NULL;
+
+ cancel = malloc(sizeof(PGcancel));
+ if (cancel == NULL)
+ return NULL;
+
+ memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
+ cancel->be_pid = conn->be_pid;
+ cancel->be_key = conn->be_key;
+ /* We use -1 to indicate an unset connection option */
+ cancel->pgtcp_user_timeout = -1;
+ cancel->keepalives = -1;
+ cancel->keepalives_idle = -1;
+ cancel->keepalives_interval = -1;
+ cancel->keepalives_count = -1;
+ if (conn->pgtcp_user_timeout != NULL)
+ {
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout,
+ &cancel->pgtcp_user_timeout,
+ conn, "tcp_user_timeout"))
+ goto fail;
+ }
+ if (conn->keepalives != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives,
+ &cancel->keepalives,
+ conn, "keepalives"))
+ goto fail;
+ }
+ if (conn->keepalives_idle != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_idle,
+ &cancel->keepalives_idle,
+ conn, "keepalives_idle"))
+ goto fail;
+ }
+ if (conn->keepalives_interval != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_interval,
+ &cancel->keepalives_interval,
+ conn, "keepalives_interval"))
+ goto fail;
+ }
+ if (conn->keepalives_count != NULL)
+ {
+ if (!pq_parse_int_param(conn->keepalives_count,
+ &cancel->keepalives_count,
+ conn, "keepalives_count"))
+ goto fail;
+ }
+
+ return cancel;
+
+fail:
+ free(cancel);
+ return NULL;
+}
+
+/* PQfreeCancel: free a cancel structure */
+void
+PQfreeCancel(PGcancel *cancel)
+{
+ free(cancel);
+}
+
+
+/*
+ * Sets an integer socket option on a TCP socket, if the provided value is
+ * not negative. Returns false if setsockopt fails for some reason.
+ *
+ * CAUTION: This needs to be signal safe, since it's used by PQcancel.
+ */
+#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
+static bool
+optional_setsockopt(int fd, int protoid, int optid, int value)
+{
+ if (value < 0)
+ return true;
+ if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
+ return false;
+ return true;
+}
+#endif
+
+
+/*
+ * PQcancel: request query cancel
+ *
+ * The return value is true if the cancel request was successfully
+ * dispatched, false if not (in which case an error message is available).
+ * Note: successful dispatch is no guarantee that there will be any effect at
+ * the backend. The application must read the operation result as usual.
+ *
+ * On failure, an error message is stored in *errbuf, which must be of size
+ * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
+ * success return.
+ *
+ * CAUTION: we want this routine to be safely callable from a signal handler
+ * (for example, an application might want to call it in a SIGINT handler).
+ * This means we cannot use any C library routine that might be non-reentrant.
+ * malloc/free are often non-reentrant, and anything that might call them is
+ * just as dangerous. We avoid sprintf here for that reason. Building up
+ * error messages with strcpy/strcat is tedious but should be quite safe.
+ * We also save/restore errno in case the signal handler support doesn't.
+ */
+int
+PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
+{
+ int save_errno = SOCK_ERRNO;
+ pgsocket tmpsock = PGINVALID_SOCKET;
+ int maxlen;
+ struct
+ {
+ uint32 packetlen;
+ CancelRequestPacket cp;
+ } crp;
+
+ if (!cancel)
+ {
+ strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
+ /* strlcpy probably doesn't change errno, but be paranoid */
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+ }
+
+ /*
+ * We need to open a temporary connection to the postmaster. Do this with
+ * only kernel calls.
+ */
+ if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
+ {
+ strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Since this connection will only be used to send a single packet of
+ * data, we don't need NODELAY. We also don't set the socket to
+ * nonblocking mode, because the API definition of PQcancel requires the
+ * cancel to be sent in a blocking way.
+ *
+ * We do set socket options related to keepalives and other TCP timeouts.
+ * This ensures that this function does not block indefinitely when
+ * reasonable keepalive and timeout settings have been provided.
+ */
+ if (cancel->raddr.addr.ss_family != AF_UNIX &&
+ cancel->keepalives != 0)
+ {
+#ifndef WIN32
+ if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+#ifdef PG_TCP_KEEPALIVE_IDLE
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
+ cancel->keepalives_idle))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPINTVL
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#ifdef TCP_KEEPCNT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
+ cancel->keepalives_count))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+
+#else /* WIN32 */
+
+#ifdef SIO_KEEPALIVE_VALS
+ if (!pqSetKeepalivesWin32(tmpsock,
+ cancel->keepalives_idle,
+ cancel->keepalives_interval))
+ {
+ strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif /* SIO_KEEPALIVE_VALS */
+#endif /* WIN32 */
+
+ /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
+#ifdef TCP_USER_TIMEOUT
+ if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
+ cancel->pgtcp_user_timeout))
+ {
+ strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+#endif
+ }
+
+retry3:
+ if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
+ cancel->raddr.salen) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry3;
+ strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /* Create and send the cancel request packet. */
+
+ crp.packetlen = pg_hton32((uint32) sizeof(crp));
+ crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ crp.cp.backendPID = pg_hton32(cancel->be_pid);
+ crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
+
+retry4:
+ if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry4;
+ strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
+ goto cancel_errReturn;
+ }
+
+ /*
+ * Wait for the postmaster to close the connection, which indicates that
+ * it's processed the request. Without this delay, we might issue another
+ * command only to find that our cancel zaps that command instead of the
+ * one we thought we were canceling. Note we don't actually expect this
+ * read to obtain any data, we are just waiting for EOF to be signaled.
+ */
+retry5:
+ if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry5;
+ /* we ignore other error conditions */
+ }
+
+ /* All done */
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return true;
+
+cancel_errReturn:
+
+ /*
+ * Make sure we don't overflow the error buffer. Leave space for the \n at
+ * the end, and for the terminating zero.
+ */
+ maxlen = errbufsize - strlen(errbuf) - 2;
+ if (maxlen >= 0)
+ {
+ /*
+ * We can't invoke strerror here, since it's not signal-safe. Settle
+ * for printing the decimal value of errno. Even that has to be done
+ * the hard way.
+ */
+ int val = SOCK_ERRNO;
+ char buf[32];
+ char *bufp;
+
+ bufp = buf + sizeof(buf) - 1;
+ *bufp = '\0';
+ do
+ {
+ *(--bufp) = (val % 10) + '0';
+ val /= 10;
+ } while (val > 0);
+ bufp -= 6;
+ memcpy(bufp, "error ", 6);
+ strncat(errbuf, bufp, maxlen);
+ strcat(errbuf, "\n");
+ }
+ if (tmpsock != PGINVALID_SOCKET)
+ closesocket(tmpsock);
+ SOCK_ERRNO_SET(save_errno);
+ return false;
+}
+
+/*
+ * PQrequestCancel: old, not thread-safe function for requesting query cancel
+ *
+ * Returns true if able to send the cancel request, false if not.
+ *
+ * On failure, the error message is saved in conn->errorMessage; this means
+ * that this can't be used when there might be other active operations on
+ * the connection object.
+ *
+ * NOTE: error messages will be cut off at the current size of the
+ * error message buffer, since we dare not try to expand conn->errorMessage!
+ */
+int
+PQrequestCancel(PGconn *conn)
+{
+ int r;
+ PGcancel *cancel;
+
+ /* Check we have an open connection */
+ if (!conn)
+ return false;
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ strlcpy(conn->errorMessage.data,
+ "PQrequestCancel() -- connection is not open\n",
+ conn->errorMessage.maxlen);
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+
+ return false;
+ }
+
+ cancel = PQgetCancel(conn);
+ if (cancel)
+ {
+ r = PQcancel(cancel, conn->errorMessage.data,
+ conn->errorMessage.maxlen);
+ PQfreeCancel(cancel);
+ }
+ else
+ {
+ strlcpy(conn->errorMessage.data, "out of memory",
+ conn->errorMessage.maxlen);
+ r = false;
+ }
+
+ if (!r)
+ {
+ conn->errorMessage.len = strlen(conn->errorMessage.data);
+ conn->errorReported = 0;
+ }
+
+ return r;
+}
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..5d08b4904d3 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -443,8 +443,6 @@ static void pgpassfileWarning(PGconn *conn);
static void default_threadlock(int acquire);
static bool sslVerifyProtocolVersion(const char *version);
static bool sslVerifyProtocolRange(const char *min, const char *max);
-static bool parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context);
/* global variable because fe-auth.c needs to access it */
@@ -2081,9 +2079,9 @@ useKeepalives(PGconn *conn)
* store it in *result, complaining if there is any trailing garbage or an
* overflow. This allows any number of leading and trailing whitespaces.
*/
-static bool
-parse_int_param(const char *value, int *result, PGconn *conn,
- const char *context)
+bool
+pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context)
{
char *end;
long numval;
@@ -2134,8 +2132,8 @@ setKeepalivesIdle(PGconn *conn)
if (conn->keepalives_idle == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ if (!pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (idle < 0)
idle = 0;
@@ -2168,8 +2166,8 @@ setKeepalivesInterval(PGconn *conn)
if (conn->keepalives_interval == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ if (!pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
if (interval < 0)
interval = 0;
@@ -2203,8 +2201,8 @@ setKeepalivesCount(PGconn *conn)
if (conn->keepalives_count == NULL)
return 1;
- if (!parse_int_param(conn->keepalives_count, &count, conn,
- "keepalives_count"))
+ if (!pq_parse_int_param(conn->keepalives_count, &count, conn,
+ "keepalives_count"))
return 0;
if (count < 0)
count = 0;
@@ -2233,8 +2231,8 @@ setKeepalivesCount(PGconn *conn)
*
* CAUTION: This needs to be signal safe, since it's used by PQcancel.
*/
-static int
-setKeepalivesWin32(pgsocket sock, int idle, int interval)
+int
+pqSetKeepalivesWin32(pgsocket sock, int idle, int interval)
{
struct tcp_keepalive ka;
DWORD retsize;
@@ -2269,15 +2267,15 @@ prepKeepalivesWin32(PGconn *conn)
int interval = -1;
if (conn->keepalives_idle &&
- !parse_int_param(conn->keepalives_idle, &idle, conn,
- "keepalives_idle"))
+ !pq_parse_int_param(conn->keepalives_idle, &idle, conn,
+ "keepalives_idle"))
return 0;
if (conn->keepalives_interval &&
- !parse_int_param(conn->keepalives_interval, &interval, conn,
- "keepalives_interval"))
+ !pq_parse_int_param(conn->keepalives_interval, &interval, conn,
+ "keepalives_interval"))
return 0;
- if (!setKeepalivesWin32(conn->sock, idle, interval))
+ if (!pqSetKeepalivesWin32(conn->sock, idle, interval))
{
libpq_append_conn_error(conn, "%s(%s) failed: error code %d",
"WSAIoctl", "SIO_KEEPALIVE_VALS",
@@ -2300,8 +2298,8 @@ setTCPUserTimeout(PGconn *conn)
if (conn->pgtcp_user_timeout == NULL)
return 1;
- if (!parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
- "tcp_user_timeout"))
+ if (!pq_parse_int_param(conn->pgtcp_user_timeout, &timeout, conn,
+ "tcp_user_timeout"))
return 0;
if (timeout < 0)
@@ -2418,8 +2416,8 @@ connectDBComplete(PGconn *conn)
*/
if (conn->connect_timeout != NULL)
{
- if (!parse_int_param(conn->connect_timeout, &timeout, conn,
- "connect_timeout"))
+ if (!pq_parse_int_param(conn->connect_timeout, &timeout, conn,
+ "connect_timeout"))
{
/* mark the connection as bad to report the parsing failure */
conn->status = CONNECTION_BAD;
@@ -2666,7 +2664,7 @@ keep_going: /* We will come back to here until there is
thisport = DEF_PGPORT;
else
{
- if (!parse_int_param(ch->port, &thisport, conn, "port"))
+ if (!pq_parse_int_param(ch->port, &thisport, conn, "port"))
goto error_return;
if (thisport < 1 || thisport > 65535)
@@ -4694,373 +4692,6 @@ PQresetPoll(PGconn *conn)
return PGRES_POLLING_FAILED;
}
-/*
- * PQgetCancel: get a PGcancel structure corresponding to a connection.
- *
- * A copy is needed to be able to cancel a running query from a different
- * thread. If the same structure is used all structure members would have
- * to be individually locked (if the entire structure was locked, it would
- * be impossible to cancel a synchronous query because the structure would
- * have to stay locked for the duration of the query).
- */
-PGcancel *
-PQgetCancel(PGconn *conn)
-{
- PGcancel *cancel;
-
- if (!conn)
- return NULL;
-
- if (conn->sock == PGINVALID_SOCKET)
- return NULL;
-
- cancel = malloc(sizeof(PGcancel));
- if (cancel == NULL)
- return NULL;
-
- memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr));
- cancel->be_pid = conn->be_pid;
- cancel->be_key = conn->be_key;
- /* We use -1 to indicate an unset connection option */
- cancel->pgtcp_user_timeout = -1;
- cancel->keepalives = -1;
- cancel->keepalives_idle = -1;
- cancel->keepalives_interval = -1;
- cancel->keepalives_count = -1;
- if (conn->pgtcp_user_timeout != NULL)
- {
- if (!parse_int_param(conn->pgtcp_user_timeout,
- &cancel->pgtcp_user_timeout,
- conn, "tcp_user_timeout"))
- goto fail;
- }
- if (conn->keepalives != NULL)
- {
- if (!parse_int_param(conn->keepalives,
- &cancel->keepalives,
- conn, "keepalives"))
- goto fail;
- }
- if (conn->keepalives_idle != NULL)
- {
- if (!parse_int_param(conn->keepalives_idle,
- &cancel->keepalives_idle,
- conn, "keepalives_idle"))
- goto fail;
- }
- if (conn->keepalives_interval != NULL)
- {
- if (!parse_int_param(conn->keepalives_interval,
- &cancel->keepalives_interval,
- conn, "keepalives_interval"))
- goto fail;
- }
- if (conn->keepalives_count != NULL)
- {
- if (!parse_int_param(conn->keepalives_count,
- &cancel->keepalives_count,
- conn, "keepalives_count"))
- goto fail;
- }
-
- return cancel;
-
-fail:
- free(cancel);
- return NULL;
-}
-
-/* PQfreeCancel: free a cancel structure */
-void
-PQfreeCancel(PGcancel *cancel)
-{
- free(cancel);
-}
-
-
-/*
- * Sets an integer socket option on a TCP socket, if the provided value is
- * not negative. Returns false if setsockopt fails for some reason.
- *
- * CAUTION: This needs to be signal safe, since it's used by PQcancel.
- */
-#if defined(TCP_USER_TIMEOUT) || !defined(WIN32)
-static bool
-optional_setsockopt(int fd, int protoid, int optid, int value)
-{
- if (value < 0)
- return true;
- if (setsockopt(fd, protoid, optid, (char *) &value, sizeof(value)) < 0)
- return false;
- return true;
-}
-#endif
-
-
-/*
- * PQcancel: request query cancel
- *
- * The return value is true if the cancel request was successfully
- * dispatched, false if not (in which case an error message is available).
- * Note: successful dispatch is no guarantee that there will be any effect at
- * the backend. The application must read the operation result as usual.
- *
- * On failure, an error message is stored in *errbuf, which must be of size
- * errbufsize (recommended size is 256 bytes). *errbuf is not changed on
- * success return.
- *
- * CAUTION: we want this routine to be safely callable from a signal handler
- * (for example, an application might want to call it in a SIGINT handler).
- * This means we cannot use any C library routine that might be non-reentrant.
- * malloc/free are often non-reentrant, and anything that might call them is
- * just as dangerous. We avoid sprintf here for that reason. Building up
- * error messages with strcpy/strcat is tedious but should be quite safe.
- * We also save/restore errno in case the signal handler support doesn't.
- */
-int
-PQcancel(PGcancel *cancel, char *errbuf, int errbufsize)
-{
- int save_errno = SOCK_ERRNO;
- pgsocket tmpsock = PGINVALID_SOCKET;
- int maxlen;
- struct
- {
- uint32 packetlen;
- CancelRequestPacket cp;
- } crp;
-
- if (!cancel)
- {
- strlcpy(errbuf, "PQcancel() -- no cancel object supplied", errbufsize);
- /* strlcpy probably doesn't change errno, but be paranoid */
- SOCK_ERRNO_SET(save_errno);
- return false;
- }
-
- /*
- * We need to open a temporary connection to the postmaster. Do this with
- * only kernel calls.
- */
- if ((tmpsock = socket(cancel->raddr.addr.ss_family, SOCK_STREAM, 0)) == PGINVALID_SOCKET)
- {
- strlcpy(errbuf, "PQcancel() -- socket() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Since this connection will only be used to send a single packet of
- * data, we don't need NODELAY. We also don't set the socket to
- * nonblocking mode, because the API definition of PQcancel requires the
- * cancel to be sent in a blocking way.
- *
- * We do set socket options related to keepalives and other TCP timeouts.
- * This ensures that this function does not block indefinitely when
- * reasonable keepalive and timeout settings have been provided.
- */
- if (cancel->raddr.addr.ss_family != AF_UNIX &&
- cancel->keepalives != 0)
- {
-#ifndef WIN32
- if (!optional_setsockopt(tmpsock, SOL_SOCKET, SO_KEEPALIVE, 1))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(SO_KEEPALIVE) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
-#ifdef PG_TCP_KEEPALIVE_IDLE
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, PG_TCP_KEEPALIVE_IDLE,
- cancel->keepalives_idle))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(" PG_TCP_KEEPALIVE_IDLE_STR ") failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPINTVL
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPINTVL,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPINTVL) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#ifdef TCP_KEEPCNT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_KEEPCNT,
- cancel->keepalives_count))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_KEEPCNT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
-
-#else /* WIN32 */
-
-#ifdef SIO_KEEPALIVE_VALS
- if (!setKeepalivesWin32(tmpsock,
- cancel->keepalives_idle,
- cancel->keepalives_interval))
- {
- strlcpy(errbuf, "PQcancel() -- WSAIoctl(SIO_KEEPALIVE_VALS) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif /* SIO_KEEPALIVE_VALS */
-#endif /* WIN32 */
-
- /* TCP_USER_TIMEOUT works the same way on Unix and Windows */
-#ifdef TCP_USER_TIMEOUT
- if (!optional_setsockopt(tmpsock, IPPROTO_TCP, TCP_USER_TIMEOUT,
- cancel->pgtcp_user_timeout))
- {
- strlcpy(errbuf, "PQcancel() -- setsockopt(TCP_USER_TIMEOUT) failed: ", errbufsize);
- goto cancel_errReturn;
- }
-#endif
- }
-
-retry3:
- if (connect(tmpsock, (struct sockaddr *) &cancel->raddr.addr,
- cancel->raddr.salen) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry3;
- strlcpy(errbuf, "PQcancel() -- connect() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /* Create and send the cancel request packet. */
-
- crp.packetlen = pg_hton32((uint32) sizeof(crp));
- crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
- crp.cp.backendPID = pg_hton32(cancel->be_pid);
- crp.cp.cancelAuthCode = pg_hton32(cancel->be_key);
-
-retry4:
- if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp))
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry4;
- strlcpy(errbuf, "PQcancel() -- send() failed: ", errbufsize);
- goto cancel_errReturn;
- }
-
- /*
- * Wait for the postmaster to close the connection, which indicates that
- * it's processed the request. Without this delay, we might issue another
- * command only to find that our cancel zaps that command instead of the
- * one we thought we were canceling. Note we don't actually expect this
- * read to obtain any data, we are just waiting for EOF to be signaled.
- */
-retry5:
- if (recv(tmpsock, (char *) &crp, 1, 0) < 0)
- {
- if (SOCK_ERRNO == EINTR)
- /* Interrupted system call - we'll just try again */
- goto retry5;
- /* we ignore other error conditions */
- }
-
- /* All done */
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return true;
-
-cancel_errReturn:
-
- /*
- * Make sure we don't overflow the error buffer. Leave space for the \n at
- * the end, and for the terminating zero.
- */
- maxlen = errbufsize - strlen(errbuf) - 2;
- if (maxlen >= 0)
- {
- /*
- * We can't invoke strerror here, since it's not signal-safe. Settle
- * for printing the decimal value of errno. Even that has to be done
- * the hard way.
- */
- int val = SOCK_ERRNO;
- char buf[32];
- char *bufp;
-
- bufp = buf + sizeof(buf) - 1;
- *bufp = '\0';
- do
- {
- *(--bufp) = (val % 10) + '0';
- val /= 10;
- } while (val > 0);
- bufp -= 6;
- memcpy(bufp, "error ", 6);
- strncat(errbuf, bufp, maxlen);
- strcat(errbuf, "\n");
- }
- if (tmpsock != PGINVALID_SOCKET)
- closesocket(tmpsock);
- SOCK_ERRNO_SET(save_errno);
- return false;
-}
-
-
-/*
- * PQrequestCancel: old, not thread-safe function for requesting query cancel
- *
- * Returns true if able to send the cancel request, false if not.
- *
- * On failure, the error message is saved in conn->errorMessage; this means
- * that this can't be used when there might be other active operations on
- * the connection object.
- *
- * NOTE: error messages will be cut off at the current size of the
- * error message buffer, since we dare not try to expand conn->errorMessage!
- */
-int
-PQrequestCancel(PGconn *conn)
-{
- int r;
- PGcancel *cancel;
-
- /* Check we have an open connection */
- if (!conn)
- return false;
-
- if (conn->sock == PGINVALID_SOCKET)
- {
- strlcpy(conn->errorMessage.data,
- "PQrequestCancel() -- connection is not open\n",
- conn->errorMessage.maxlen);
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
-
- return false;
- }
-
- cancel = PQgetCancel(conn);
- if (cancel)
- {
- r = PQcancel(cancel, conn->errorMessage.data,
- conn->errorMessage.maxlen);
- PQfreeCancel(cancel);
- }
- else
- {
- strlcpy(conn->errorMessage.data, "out of memory",
- conn->errorMessage.maxlen);
- r = false;
- }
-
- if (!r)
- {
- conn->errorMessage.len = strlen(conn->errorMessage.data);
- conn->errorReported = 0;
- }
-
- return r;
-}
-
-
/*
* pqPacketSend() -- convenience routine to send a message to server.
*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..48c10b474f5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -678,12 +678,18 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
+ const char *context);
extern pgthreadlock_t pg_g_threadlock;
#define pglock_thread() pg_g_threadlock(true)
#define pgunlock_thread() pg_g_threadlock(false)
+#if defined(WIN32) && defined(SIO_KEEPALIVE_VALS)
+extern int pqSetKeepalivesWin32(pgsocket sock, int idle, int interval);
+#endif
+
/* === in fe-exec.c === */
extern void pqSetResultError(PGresult *res, PQExpBuffer errorMessage, int offset);
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index c76a1e40c83..a47b6f425dd 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -6,6 +6,7 @@
libpq_sources = files(
'fe-auth-scram.c',
'fe-auth.c',
+ 'fe-cancel.c',
'fe-connect.c',
'fe-exec.c',
'fe-lobj.c',
base-commit: a3a836fb5e51183eae624d43225279306c2285b8
--
2.34.1
[application/x-patch] v28-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/5-v28-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From cb5b87e6e0c9127352013453bc2e944696d0925a Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v28 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/x-patch] v28-0004-Add-non-blocking-version-of-PQcancel.patch (42.8K, ../../CAGECzQR=7S6EuYi8rOH+6J3UWbbCip3F_11GJrWF-D6D6tPgrg@mail.gmail.com/6-v28-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From 819ecc80382b93ffa0a9757119c396e4bf667908 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v28 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 +++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 284 ++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++++++-
7 files changed, 964 insertions(+), 38 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index ce28d39f3f5..e37ee0c45ec 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pq_release_conn_hosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index aeb3adc0e31..2cd95767fdb 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2354,10 +2402,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2499,7 +2555,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2624,13 +2683,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3272,6 +3335,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -4021,8 +4107,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4390,6 +4482,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pq_release_conn_hosts(conn);
free(conn->client_encoding_initial);
@@ -4540,6 +4633,15 @@ pq_release_conn_hosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4593,7 +4695,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c1ff12dd396..e780b62b2bd 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -678,6 +687,7 @@ extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pq_parse_int_param(const char *value, int *result, PGconn *conn,
const char *context);
extern void pq_release_conn_hosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-29 11:44 Alvaro Herrera <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 36+ messages in thread
From: Alvaro Herrera @ 2024-01-29 11:44 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On 2024-Jan-28, Jelte Fennema-Nio wrote:
> On Sun, 28 Jan 2024 at 10:51, Jelte Fennema-Nio <[email protected]> wrote:
> > Both of those are fixed now.
>
> Okay, there turned out to also be an issue on Windows with
> setKeepalivesWin32 not being available in fe-cancel.c. That's fixed
> now too (as well as some minor formatting issues).
Thanks! I committed 0001 now. I also renamed the new
pq_parse_int_param to pqParseIntParam, for consistency with other
routines there. Please rebase the other patches.
Thanks,
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Thou shalt check the array bounds of all strings (indeed, all arrays), for
surely where thou typest "foo" someone someday shall type
"supercalifragilisticexpialidocious" (5th Commandment for C programmers)
^ permalink raw reply [nested|flat] 36+ messages in thread
* Re: [EXTERNAL] Re: Add non-blocking version of PQcancel
@ 2024-01-29 12:28 Jelte Fennema-Nio <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-29 12:28 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Thomas Munro <[email protected]>; Denis Laxalde <[email protected]>; Greg Stark <[email protected]>; Tom Lane <[email protected]>; Gregory Stark (as CFM) <[email protected]>; Jelte Fennema <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Justin Pryzby <[email protected]>; Robert Haas <[email protected]>; [email protected] <[email protected]>
On Mon, 29 Jan 2024 at 12:44, Alvaro Herrera <[email protected]> wrote:
> Thanks! I committed 0001 now. I also renamed the new
> pq_parse_int_param to pqParseIntParam, for consistency with other
> routines there. Please rebase the other patches.
Awesome! Rebased, and renamed pq_release_conn_hosts to
pqReleaseConnHosts for the same consistency reasons.
Attachments:
[application/octet-stream] v29-0004-Add-non-blocking-version-of-PQcancel.patch (42.8K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/2-v29-0004-Add-non-blocking-version-of-PQcancel.patch)
download | inline diff:
From c24759d9932d6cf5d9e18c30105e1bb520270de9 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:00 +0100
Subject: [PATCH v29 4/5] Add non-blocking version of PQcancel
This patch makes the following changes in libpq:
1. Add a new PQcancelSend function, which sends cancellation requests
using the regular connection establishment code. This makes sure
that cancel requests support and use all connection options
including encryption.
2. Add a new PQcancelConn function which allows sending cancellation in
a non-blocking way by using it together with the newly added
PQcancelPoll and PQcancelSocket.
The existing PQcancel API is using blocking IO. This makes PQcancel
impossible to use in an event loop based codebase, without blocking the
event loop until the call returns. PQcancelConn can now be used instead,
to have a non-blocking way of sending cancel requests.
This patch also includes a test for all of libpq cancellation APIs. The
test can be easily run like this:
cd src/test/modules/libpq_pipeline
make && ./libpq_pipeline cancel
---
doc/src/sgml/libpq.sgml | 280 +++++++++++++++--
src/interfaces/libpq/exports.txt | 8 +
src/interfaces/libpq/fe-cancel.c | 284 ++++++++++++++++++
src/interfaces/libpq/fe-connect.c | 130 +++++++-
src/interfaces/libpq/libpq-fe.h | 27 +-
src/interfaces/libpq/libpq-int.h | 10 +
.../modules/libpq_pipeline/libpq_pipeline.c | 263 +++++++++++++++-
7 files changed, 964 insertions(+), 38 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d0d5aefadc0..9808e678650 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -265,7 +265,7 @@ PGconn *PQsetdb(char *pghost,
<varlistentry id="libpq-PQconnectStartParams">
<term><function>PQconnectStartParams</function><indexterm><primary>PQconnectStartParams</primary></indexterm></term>
<term><function>PQconnectStart</function><indexterm><primary>PQconnectStart</primary></indexterm></term>
- <term><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
+ <term id="libpq-PQconnectPoll"><function>PQconnectPoll</function><indexterm><primary>PQconnectPoll</primary></indexterm></term>
<listitem>
<para>
<indexterm><primary>nonblocking connection</primary></indexterm>
@@ -5281,7 +5281,7 @@ int PQisBusy(PGconn *conn);
<xref linkend="libpq-PQsendQuery"/>/<xref linkend="libpq-PQgetResult"/>
can also attempt to cancel a command that is still being processed
by the server; see <xref linkend="libpq-cancel"/>. But regardless of
- the return value of <xref linkend="libpq-PQcancel"/>, the application
+ the return value of <xref linkend="libpq-PQcancelSend"/>, the application
must continue with the normal result-reading sequence using
<xref linkend="libpq-PQgetResult"/>. A successful cancellation will
simply cause the command to terminate sooner than it would have
@@ -6034,13 +6034,223 @@ int PQsetSingleRowMode(PGconn *conn);
this section.
<variablelist>
+ <varlistentry id="libpq-PQcancelConn">
+ <term><function>PQcancelConn</function><indexterm><primary>PQcancelConn</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Prepares a connection over which a cancel request can be sent.
+<synopsis>
+PGcancelConn *PQcancelConn(PGconn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQcancelConn"/> creates a
+ <structname>PGcancelConn</structname><indexterm><primary>PGcancelConn</primary></indexterm>
+ object, but it won't instantly start sending a cancel request over this
+ connection. A cancel request can be sent over this connection in a
+ blocking manner using <xref linkend="libpq-PQcancelSend"/> and in a
+ non-blocking manner using <xref linkend="libpq-PQcancelPoll"/>.
+ The return value should can be passed to <xref linkend="libpq-PQcancelStatus"/>,
+ to check if the <structname>PGcancelConn</structname> object was
+ created successfully. The <structname>PGcancelConn</structname> object
+ is an opaque structure that is not meant to be accessed directly by the
+ application. This <structname>PGcancelConn</structname> object can be
+ used to cancel the query that's running on the original connection in a
+ thread-safe way.
+ </para>
+
+ <para>
+ If the original connection is encrypted (using TLS or GSS), then the
+ connection for the cancel request is encrypted in the same way. Any
+ connection options that are only used during authentication or after
+ authentication of the client are ignored though, because cancellation
+ requests do not require authentication and the connection is closed right
+ after the cancellation request is submitted.
+ </para>
+
+ <para>
+ Note that when <function>PQcancelConn</function> returns a non-null
+ pointer, you must call <xref linkend="libpq-PQcancelFinish"/> when you
+ are finished with it, in order to dispose of the structure and any
+ associated memory blocks. This must be done even if the cancel request
+ failed or was abandoned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSend">
+ <term><function>PQcancelSend</function><indexterm><primary>PQcancelSend</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Requests that the server abandons processing of the current command in a blocking manner.
+<synopsis>
+int PQcancelSend(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ The request is made over the given <structname>PGcancelConn</structname>,
+ which needs to be created with <xref linkend="libpq-PQcancelConn"/>
+ The return value of <xref linkend="libpq-PQcancelSend"/>
+ is 1 if the cancel request was successfully
+ dispatched and 0 if not. If it was unsuccessful, the error message can be
+ retrieved using <xref linkend="libpq-PQcancelErrorMessage"/>.
+ </para>
+
+ <para>
+ Successful dispatch of the cancellation is no guarantee that the request
+ will have any effect, however. If the cancellation is effective, the
+ command being canceled will terminate early and return an error result.
+ If the cancellation fails (say, because the server was already done
+ processing the command), then there will be no visible result at all.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelStatus">
+ <term><function>PQcancelStatus</function><indexterm><primary>PQcancelStatus</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQstatus"/> that can be used for
+ cancellation connections.
+<synopsis>
+ConnStatusType PQcancelStatus(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ <para>
+ In addition to all the statuses that a <structname>PGconn</structname>
+ can have, this connection can have one additional status:
+
+ <variablelist>
+ <varlistentry id="libpq-connection-starting">
+ <term><symbol>CONNECTION_STARTING</symbol></term>
+ <listitem>
+ <para>
+ Waiting for the first call to <xref linkend="libpq-PQcancelPoll"/>,
+ to actually open the socket. This is the connection state right after
+ calling <xref linkend="libpq-PQcancelConn"/>. No connection to the
+ server has been initiated yet at this point. To actually start
+ sending the cancel request use <xref linkend="libpq-PQcancelPoll"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ One final note about the returned statuses is that
+ <symbol>CONNECTION_OK</symbol> has a slightly different meaning for a
+ <structname>PGcancelConn</structname> than what it has for a
+ <structname>PGconn</structname>. When <xref linkend="libpq-PQcancelStatus"/>
+ returns <symbol>CONNECTION_OK</symbol> for a <structname>PGcancelConn</structname>
+ it means that that the dispatch of the cancel request has completed (although
+ this is no promise that the query was actually canceled) and that the
+ connection is now closed. While a <symbol>CONNECTION_OK</symbol> result
+ for <structname>PGconn</structname> means that queries can be sent over
+ the connection.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelSocket">
+ <term><function>PQcancelSocket</function><indexterm><primary>PQcancelSocket</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQsocket"/> that can be used for
+ cancellation connections.
+<synopsis>
+int PQcancelSocket(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelPoll">
+ <term><function>PQcancelPoll</function><indexterm><primary>PQcancelPoll</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQconnectPoll"/> that can be used for
+ cancellation connections.
+<synopsis>
+PostgresPollingStatusType PQcancelPoll(PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelErrorMessage">
+ <term><function>PQcancelErrorMessage</function><indexterm><primary>PQcancelErrorMessage</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ A version of <xref linkend="libpq-PQerrorMessage"/> that can be used for
+ cancellation connections.
+<synopsis>
+char *PQcancelErrorMessage(const PGcancelConn *conn);
+</synopsis>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelFinish">
+ <term><function>PQcancelFinish</function><indexterm><primary>PQcancelFinish</primary></indexterm></term>
+ <listitem>
+ <para>
+ Closes the cancel connection (if it did not finish sending the cancel
+ request yet). Also frees memory used by the <structname>PGcancelConn</structname>
+ object.
+<synopsis>
+void PQcancelFinish(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ Note that even if the cancel attempt fails (as
+ indicated by <xref linkend="libpq-PQcancelStatus"/>), the application should call <xref linkend="libpq-PQcancelFinish"/>
+ to free the memory used by the <structname>PGcancelConn</structname> object.
+ The <structname>PGcancelConn</structname> pointer must not be used again after
+ <xref linkend="libpq-PQcancelFinish"/> has been called.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-PQcancelReset">
+ <term><function>PQcancelReset</function><indexterm><primary>PQcancelReset</primary></indexterm></term>
+ <listitem>
+ <para>
+ Resets the <symbol>PGcancelConn</symbol> so it can be reused for a new
+ cancel connection.
+<synopsis>
+void PQcancelReset(PGcancelConn *conn);
+</synopsis>
+ </para>
+
+ <para>
+ If the <symbol>PGcancelConn</symbol> is currently used to send a cancel
+ request, then this connection is closed. It will then prepare the
+ <symbol>PGcancelConn</symbol> object such that it can be used to send a
+ new cancel request. This can be used to create one <symbol>PGcancelConn</symbol>
+ for a <symbol>PGconn</symbol> and reuse that multiple times throughout
+ the lifetime of the original <symbol>PGconn</symbol>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQgetCancel">
<term><function>PQgetCancel</function><indexterm><primary>PQgetCancel</primary></indexterm></term>
<listitem>
<para>
Creates a data structure containing the information needed to cancel
- a command issued through a particular database connection.
+ a command using <xref linkend="libpq-PQcancel"/>.
<synopsis>
PGcancel *PQgetCancel(PGconn *conn);
</synopsis>
@@ -6082,14 +6292,28 @@ void PQfreeCancel(PGcancel *cancel);
<listitem>
<para>
- Requests that the server abandon processing of the current command.
+ An insecure version of <xref linkend="libpq-PQcancelSend"/>, but one
+ that can be used safely from within a signal handler.
<synopsis>
int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</synopsis>
</para>
<para>
- The return value is 1 if the cancel request was successfully
+ <xref linkend="libpq-PQcancel"/> should only be used if it's necessary
+ to cancel a query from a signal-handler. If signal-safety is not needed,
+ <xref linkend="libpq-PQcancelSend"/> should be used to cancel the query
+ instead. <xref linkend="libpq-PQcancel"/> can be safely invoked from a
+ signal handler, if the <parameter>errbuf</parameter> is a local variable
+ in the signal handler. The <structname>PGcancel</structname> object is
+ read-only as far as <xref linkend="libpq-PQcancel"/> is concerned, so it
+ can also be invoked from a thread that is separate from the one
+ manipulating the <structname>PGconn</structname> object.
+ </para>
+
+ <para>
+ The return value of <xref linkend="libpq-PQcancel"/>
+ is 1 if the cancel request was successfully
dispatched and 0 if not. If not, <parameter>errbuf</parameter> is filled
with an explanatory error message. <parameter>errbuf</parameter>
must be a char array of size <parameter>errbufsize</parameter> (the
@@ -6097,21 +6321,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
</para>
<para>
- Successful dispatch is no guarantee that the request will have
- any effect, however. If the cancellation is effective, the current
- command will terminate early and return an error result. If the
- cancellation fails (say, because the server was already done
- processing the command), then there will be no visible result at
- all.
- </para>
-
- <para>
- <xref linkend="libpq-PQcancel"/> can safely be invoked from a signal
- handler, if the <parameter>errbuf</parameter> is a local variable in the
- signal handler. The <structname>PGcancel</structname> object is read-only
- as far as <xref linkend="libpq-PQcancel"/> is concerned, so it can
- also be invoked from a thread that is separate from the one
- manipulating the <structname>PGconn</structname> object.
+ To achieve signal-safety, some concessions needed to be made in the
+ implementation of <xref linkend="libpq-PQcancel"/>. Not all connection
+ options of the original connection are used when establishing a
+ connection for the cancellation request. This function connects to
+ postgres on the same address and port as the original connection. The
+ only connection options that are honored during this connection are
+ <varname>keepalives</varname>,
+ <varname>keepalives_idle</varname>,
+ <varname>keepalives_interval</varname>,
+ <varname>keepalives_count</varname>, and
+ <varname>tcp_user_timeout</varname>.
+ So, for example
+ <varname>connect_timeout</varname>,
+ <varname>gssencmode</varname>, and
+ <varname>sslmode</varname> are ignored. <emphasis>This means the connection
+ for the cancel request is never encrypted using TLS or GSS</emphasis>.
</para>
</listitem>
</varlistentry>
@@ -6123,13 +6348,22 @@ int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
<listitem>
<para>
- <xref linkend="libpq-PQrequestCancel"/> is a deprecated variant of
- <xref linkend="libpq-PQcancel"/>.
+ <xref linkend="libpq-PQrequestCancel"/> is a deprecated and insecure
+ variant of <xref linkend="libpq-PQcancelSend"/>.
<synopsis>
int PQrequestCancel(PGconn *conn);
</synopsis>
</para>
+ <para>
+ <xref linkend="libpq-PQrequestCancel"/> only exists because of backwards
+ compatibility reasons. <xref linkend="libpq-PQcancelSend"/> should be
+ used instead, to avoid the security and thread-safety issues that this
+ function has. This function has the same security issues as
+ <xref linkend="libpq-PQcancel"/>, but without the benefit of being
+ signal-safe.
+ </para>
+
<para>
Requests that the server abandon processing of the current
command. It operates directly on the
@@ -9356,7 +9590,7 @@ int PQisthreadsafe();
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
used in multithread programs. <xref linkend="libpq-PQrequestCancel"/>
- can be replaced by <xref linkend="libpq-PQcancel"/>.
+ can be replaced by <xref linkend="libpq-PQcancelSend"/>.
<xref linkend="libpq-PQoidStatus"/> can be replaced by
<xref linkend="libpq-PQoidValue"/>.
</para>
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb16..125bc80679a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,11 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQcancelSend 194
+PQcancelConn 195
+PQcancelPoll 196
+PQcancelStatus 197
+PQcancelSocket 198
+PQcancelErrorMessage 199
+PQcancelReset 200
+PQcancelFinish 201
diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c
index 51f8d8a78c4..7416791d9f0 100644
--- a/src/interfaces/libpq/fe-cancel.c
+++ b/src/interfaces/libpq/fe-cancel.c
@@ -21,6 +21,290 @@
#include "libpq-int.h"
#include "port/pg_bswap.h"
+
+/*
+ * PQcancelConn
+ *
+ * Asynchronously cancel a query on the given connection. This requires polling
+ * the returned PGcancelConn to actually complete the cancellation of the
+ * query.
+ */
+PGcancelConn *
+PQcancelConn(PGconn *conn)
+{
+ PGconn *cancelConn = pqMakeEmptyPGconn();
+ pg_conn_host originalHost;
+
+ if (cancelConn == NULL)
+ return NULL;
+
+ /* Check we have an open connection */
+ if (!conn)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection was NULL");
+ return (PGcancelConn *) cancelConn;
+ }
+
+ if (conn->sock == PGINVALID_SOCKET)
+ {
+ libpq_append_conn_error(cancelConn, "passed connection is not open");
+ return (PGcancelConn *) cancelConn;
+ }
+
+
+ /*
+ * Indicate that this connection is used to send a cancellation
+ */
+ cancelConn->cancelRequest = true;
+
+ if (!pqCopyPGconn(conn, cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Compute derived options
+ */
+ if (!pqConnectOptions2(cancelConn))
+ return (PGcancelConn *) cancelConn;
+
+ /*
+ * Copy cancellation token data from the original connnection
+ */
+ cancelConn->be_pid = conn->be_pid;
+ cancelConn->be_key = conn->be_key;
+
+ /*
+ * Cancel requests should not iterate over all possible hosts. The request
+ * needs to be sent to the exact host and address that the original
+ * connection used. So we manually create the host and address arrays with
+ * a single element after freeing the host array that we generated from
+ * the connection options.
+ */
+ pqReleaseConnHosts(cancelConn);
+ cancelConn->nconnhost = 1;
+ cancelConn->naddr = 1;
+
+ cancelConn->connhost = calloc(cancelConn->nconnhost, sizeof(pg_conn_host));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ originalHost = conn->connhost[conn->whichhost];
+ if (originalHost.host)
+ {
+ cancelConn->connhost[0].host = strdup(originalHost.host);
+ if (!cancelConn->connhost[0].host)
+ goto oom_error;
+ }
+ if (originalHost.hostaddr)
+ {
+ cancelConn->connhost[0].hostaddr = strdup(originalHost.hostaddr);
+ if (!cancelConn->connhost[0].hostaddr)
+ goto oom_error;
+ }
+ if (originalHost.port)
+ {
+ cancelConn->connhost[0].port = strdup(originalHost.port);
+ if (!cancelConn->connhost[0].port)
+ goto oom_error;
+ }
+ if (originalHost.password)
+ {
+ cancelConn->connhost[0].password = strdup(originalHost.password);
+ if (!cancelConn->connhost[0].password)
+ goto oom_error;
+ }
+
+ cancelConn->addr = calloc(cancelConn->naddr, sizeof(AddrInfo));
+ if (!cancelConn->connhost)
+ goto oom_error;
+
+ cancelConn->addr[0].addr = conn->raddr;
+ cancelConn->addr[0].family = conn->raddr.addr.ss_family;
+
+ cancelConn->status = CONNECTION_STARTING;
+ return (PGcancelConn *) cancelConn;
+
+oom_error:
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(cancelConn, "out of memory");
+ return (PGcancelConn *) cancelConn;
+}
+
+
+/*
+ * PQcancelSend
+ *
+ * Send a cancellation request in a blocking fashion.
+ * Returns 1 if successful 0 if not.
+ */
+int
+PQcancelSend(PGcancelConn * cancelConn)
+{
+ if (!cancelConn || cancelConn->conn.status == CONNECTION_BAD)
+ return 1;
+
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_BAD;
+ return 1;
+ }
+
+ return pqConnectDBComplete(&cancelConn->conn);
+}
+
+/*
+ * PQcancelPoll
+ *
+ * Poll a cancel connection. For usage details see PQconnectPoll.
+ */
+PostgresPollingStatusType
+PQcancelPoll(PGcancelConn * cancelConn)
+{
+ PGconn *conn = (PGconn *) cancelConn;
+ int n;
+
+ /*
+ * Before we can call PQconnectPoll we first need to start the connection
+ * using pqConnectDBStart. Non-cancel connections already do this whenever
+ * the connection is initialized. But cancel connections wait until the
+ * caller starts polling, because there might be a large delay between
+ * creating a cancel connection and actually wanting to use it.
+ */
+ if (conn->status == CONNECTION_STARTING)
+ {
+ if (!pqConnectDBStart(&cancelConn->conn))
+ {
+ cancelConn->conn.status = CONNECTION_STARTED;
+ return PGRES_POLLING_WRITING;
+ }
+ }
+
+ /*
+ * The rest of the connection establishement we leave to PQconnectPoll,
+ * since it's very similar to normal connection establishment. But once we
+ * get to the CONNECTION_AWAITING_RESPONSE we need to do our own thing.
+ */
+ if (conn->status != CONNECTION_AWAITING_RESPONSE)
+ {
+ return PQconnectPoll(conn);
+ }
+
+ /*
+ * At this point we are waiting on the server to close the connection,
+ * which is its way of communicating that the cancel has been handled.
+ */
+
+ n = pqReadData(conn);
+
+ if (n == 0)
+ return PGRES_POLLING_READING;
+
+#ifndef WIN32
+
+ /*
+ * If we receive an error report it, but only if errno is non-zero.
+ * Otherwise we assume it's an EOF, which is what we expect from the
+ * server.
+ *
+ * We skip this for Windows, because Windows is a bit special in its EOF
+ * behaviour for TCP. Sometimes it will error with an ECONNRESET when
+ * there is a clean connection closure. See these threads for details:
+ * https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
+ *
+ * https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
+ *
+ * PQcancel ignores such errors and reports success for the cancellation
+ * anyway, so even if this is not always correct we do the same here.
+ */
+ if (n < 0 && errno != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+#endif
+
+ /*
+ * We don't expect any data, only connection closure. So if we strangly do
+ * receive some data we consider that an error.
+ */
+ if (n > 0)
+ {
+
+ libpq_append_conn_error(conn, "received unexpected response from server");
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ /*
+ * Getting here means that we received an EOF. Which is what we were
+ * expecting. The cancel request has completed.
+ */
+ cancelConn->conn.status = CONNECTION_OK;
+ resetPQExpBuffer(&conn->errorMessage);
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * PQcancelStatus
+ *
+ * Get the status of a cancel connection.
+ */
+ConnStatusType
+PQcancelStatus(const PGcancelConn * cancelConn)
+{
+ return PQstatus((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelSocket
+ *
+ * Get the socket of the cancel connection.
+ */
+int
+PQcancelSocket(const PGcancelConn * cancelConn)
+{
+ return PQsocket((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelErrorMessage
+ *
+ * Get the socket of the cancel connection.
+ */
+char *
+PQcancelErrorMessage(const PGcancelConn * cancelConn)
+{
+ return PQerrorMessage((const PGconn *) cancelConn);
+}
+
+/*
+ * PQcancelReset
+ *
+ * Resets the cancel connection, so it can be reused to send a new cancel
+ * request.
+ */
+void
+PQcancelReset(PGcancelConn * cancelConn)
+{
+ pqClosePGconn((PGconn *) cancelConn);
+ cancelConn->conn.status = CONNECTION_STARTING;
+ cancelConn->conn.whichhost = 0;
+ cancelConn->conn.whichaddr = 0;
+ cancelConn->conn.try_next_host = false;
+ cancelConn->conn.try_next_addr = false;
+}
+
+/*
+ * PQcancelFinish
+ *
+ * Closes and frees the cancel connection.
+ */
+void
+PQcancelFinish(PGcancelConn * cancelConn)
+{
+ PQfinish((PGconn *) cancelConn);
+}
+
+
/*
* PQgetCancel: get a PGcancel structure corresponding to a connection.
*
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index dd240d42d70..4add35ec5cf 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -616,8 +616,17 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+
+ /*
+ * Cancel connections should save their be_pid and be_key across
+ * PQcancelReset invocations. Otherwise they would not have access to the
+ * secret token of the connection they are supposed to cancel anymore.
+ */
+ if (!conn->cancelRequest)
+ {
+ conn->be_pid = 0;
+ conn->be_key = 0;
+ }
}
@@ -923,6 +932,45 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
return true;
}
+/*
+ * Copy over option values from srcConn to dstConn
+ *
+ * Don't put anything cute here --- intelligence should be in
+ * connectOptions2 ...
+ *
+ * Returns true on success. On failure, returns false and sets error message of
+ * dstConn.
+ */
+bool
+pqCopyPGconn(PGconn *srcConn, PGconn *dstConn)
+{
+ const internalPQconninfoOption *option;
+
+ /* copy over connection options */
+ for (option = PQconninfoOptions; option->keyword; option++)
+ {
+ if (option->connofs >= 0)
+ {
+ const char **tmp = (const char **) ((char *) srcConn + option->connofs);
+
+ if (*tmp)
+ {
+ char **dstConnmember = (char **) ((char *) dstConn + option->connofs);
+
+ if (*dstConnmember)
+ free(*dstConnmember);
+ *dstConnmember = strdup(*tmp);
+ if (*dstConnmember == NULL)
+ {
+ libpq_append_conn_error(dstConn, "out of memory");
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+}
+
/*
* connectOptions1
*
@@ -2308,10 +2356,18 @@ pqConnectDBStart(PGconn *conn)
* Set up to try to connect to the first host. (Setting whichhost = -1 is
* a bit of a cheat, but PQconnectPoll will advance it to 0 before
* anything else looks at it.)
+ *
+ * Cancel requests are special though, they should only try one host and
+ * address. These fields have already set up in PQcancelConn. So leave
+ * these fields alone for cancel requests.
*/
- conn->whichhost = -1;
- conn->try_next_addr = false;
- conn->try_next_host = true;
+ if (!conn->cancelRequest)
+ {
+ conn->whichhost = -1;
+ conn->try_next_host = true;
+ conn->try_next_addr = false;
+ }
+
conn->status = CONNECTION_NEEDED;
/* Also reset the target_server_type state if needed */
@@ -2453,7 +2509,10 @@ pqConnectDBComplete(PGconn *conn)
/*
* Now try to advance the state machine.
*/
- flag = PQconnectPoll(conn);
+ if (conn->cancelRequest)
+ flag = PQcancelPoll((PGcancelConn *) conn);
+ else
+ flag = PQconnectPoll(conn);
}
}
@@ -2578,13 +2637,17 @@ keep_going: /* We will come back to here until there is
* Oops, no more hosts.
*
* If we are trying to connect in "prefer-standby" mode, then drop
- * the standby requirement and start over.
+ * the standby requirement and start over. Don't do this for
+ * cancel requests though, since we are certain the list of
+ * servers won't change as the target_server_type option is not
+ * applicable to those connections.
*
* Otherwise, an appropriate error message is already set up, so
* we just need to set the right status.
*/
if (conn->target_server_type == SERVER_TYPE_PREFER_STANDBY &&
- conn->nconnhost > 0)
+ conn->nconnhost > 0 &&
+ !conn->cancelRequest)
{
conn->target_server_type = SERVER_TYPE_PREFER_STANDBY_PASS2;
conn->whichhost = 0;
@@ -3226,6 +3289,29 @@ keep_going: /* We will come back to here until there is
}
#endif /* USE_SSL */
+ /*
+ * For cancel requests this is as far as we need to go in the
+ * connection establishment. Now we can actually send our
+ * cancellation request.
+ */
+ if (conn->cancelRequest)
+ {
+ CancelRequestPacket cancelpacket;
+
+ packetlen = sizeof(cancelpacket);
+ cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE);
+ cancelpacket.backendPID = pg_hton32(conn->be_pid);
+ cancelpacket.cancelAuthCode = pg_hton32(conn->be_key);
+ if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK)
+ {
+ libpq_append_conn_error(conn, "could not send cancel packet: %s",
+ SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
+ goto error_return;
+ }
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ return PGRES_POLLING_READING;
+ }
+
/*
* Build the startup packet.
*/
@@ -3975,8 +4061,14 @@ keep_going: /* We will come back to here until there is
}
}
- /* We can release the address list now. */
- release_conn_addrinfo(conn);
+ /*
+ * For non cancel requests we can release the address list
+ * now. For cancel requests we never actually resolve
+ * addresses and instead the addrinfo exists for the lifetime
+ * of the connection.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/*
* Contents of conn->errorMessage are no longer interesting
@@ -4344,6 +4436,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
+ release_conn_addrinfo(conn);
pqReleaseConnHosts(conn);
free(conn->client_encoding_initial);
@@ -4494,6 +4587,15 @@ pqReleaseConnHosts(PGconn *conn)
static void
sendTerminateConn(PGconn *conn)
{
+ /*
+ * The Postgres cancellation protocol does not have a notion of a
+ * Terminate message, so don't send one.
+ */
+ if (conn->cancelRequest)
+ {
+ return;
+ }
+
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -4547,7 +4649,13 @@ pqClosePGconn(PGconn *conn)
conn->pipelineStatus = PQ_PIPELINE_OFF;
pqClearAsyncResult(conn); /* deallocate result */
pqClearConnErrorState(conn);
- release_conn_addrinfo(conn);
+
+ /*
+ * Since cancel requests never change their addrinfo we don't free it
+ * here. Otherwise we would have to rebuild it during a PQcancelReset.
+ */
+ if (!conn->cancelRequest)
+ release_conn_addrinfo(conn);
/* Reset all state obtained from server, too */
pqDropServerData(conn);
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..857ba54d943 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_STARTING /* Waiting for connection attempt to be
+ * started. */
} ConnStatusType;
typedef enum
@@ -165,6 +167,11 @@ typedef enum
*/
typedef struct pg_conn PGconn;
+/* PGcancelConn encapsulates a cancel connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+typedef struct pg_cancel_conn PGcancelConn;
+
/* PGresult encapsulates the result of a query (or more precisely, of a single
* SQL command --- a query string given to PQsendQuery can contain multiple
* commands and thus return multiple PGresult objects).
@@ -321,16 +328,30 @@ extern PostgresPollingStatusType PQresetPoll(PGconn *conn);
/* Synchronous (blocking) */
extern void PQreset(PGconn *conn);
+/* Create a PGcancelConn that's used to cancel a query on the given PGconn */
+extern PGcancelConn * PQcancelConn(PGconn *conn);
+/* issue a blocking cancel request */
+extern int PQcancelSend(PGcancelConn * conn);
+
+/* issue or poll a non-blocking cancel request */
+extern PostgresPollingStatusType PQcancelPoll(PGcancelConn * cancelConn);
+extern ConnStatusType PQcancelStatus(const PGcancelConn * cancelConn);
+extern int PQcancelSocket(const PGcancelConn * cancelConn);
+extern char *PQcancelErrorMessage(const PGcancelConn * cancelConn);
+extern void PQcancelReset(PGcancelConn * cancelConn);
+extern void PQcancelFinish(PGcancelConn * cancelConn);
+
+
/* request a cancel structure */
extern PGcancel *PQgetCancel(PGconn *conn);
/* free a cancel structure */
extern void PQfreeCancel(PGcancel *cancel);
-/* issue a cancel request */
+/* a less secure version of PQcancelSend, but one which is signal-safe */
extern int PQcancel(PGcancel *cancel, char *errbuf, int errbufsize);
-/* backwards compatible version of PQcancel; not thread-safe */
+/* deprecated version of PQcancel; not thread-safe */
extern int PQrequestCancel(PGconn *conn);
/* Accessor functions for PGconn objects */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 07732927a5b..be45d6098a5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -409,6 +409,10 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ bool cancelRequest; /* true if this connection is used to send a
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -621,6 +625,11 @@ struct pg_conn
PQExpBufferData workBuffer; /* expansible string */
};
+struct pg_cancel_conn
+{
+ PGconn conn;
+};
+
/* PGcancel stores all data necessary to cancel a connection. A copy of this
* data is required to safely cancel a connection running on a different
* thread.
@@ -681,6 +690,7 @@ extern int pqSetKeepalivesWin32(pgsocket sock, int idle, int interval);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
extern void pqReleaseConnHosts(PGconn *conn);
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 5f43aa40de4..580003002e4 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -86,6 +86,264 @@ pg_fatal_impl(int line, const char *fmt,...)
exit(1);
}
+/*
+ * Check that the query on the given connection got canceled.
+ *
+ * This is a function wrapped in a macro to make the reported line number
+ * in an error match the line number of the invocation.
+ */
+#define confirm_query_canceled(conn) confirm_query_canceled_impl(__LINE__, conn)
+static void
+confirm_query_canceled_impl(int line, PGconn *conn)
+{
+ PGresult *res = NULL;
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ pg_fatal_impl(line, "PQgetResult returned null: %s",
+ PQerrorMessage(conn));
+ if (PQresultStatus(res) != PGRES_FATAL_ERROR)
+ pg_fatal_impl(line, "query did not fail when it was expected");
+ if (strcmp(PQresultErrorField(res, PG_DIAG_SQLSTATE), "57014") != 0)
+ pg_fatal_impl(line, "query failed with a different error than cancellation: %s",
+ PQerrorMessage(conn));
+ PQclear(res);
+ while (PQisBusy(conn))
+ {
+ PQconsumeInput(conn);
+ }
+}
+
+#define send_cancellable_query(conn, monitorConn) send_cancellable_query_impl(__LINE__, conn, monitorConn)
+static void
+send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn)
+{
+ const char *env_wait;
+ const Oid paramTypes[1] = {INT4OID};
+
+ env_wait = getenv("PG_TEST_TIMEOUT_DEFAULT");
+ if (env_wait == NULL)
+ env_wait = "180";
+
+ if (PQsendQueryParams(conn, "SELECT pg_sleep($1)", 1, paramTypes, &env_wait, NULL, NULL, 0) != 1)
+ pg_fatal_impl(line, "failed to send query: %s", PQerrorMessage(conn));
+
+ /*
+ * Wait until the query is actually running. Otherwise sending a
+ * cancellation request might not cancel the query due to race conditions.
+ */
+ while (true)
+ {
+ char *value = NULL;
+ PGresult *res = PQexec(
+ monitorConn,
+ "SELECT count(*) FROM pg_stat_activity WHERE "
+ "query = 'SELECT pg_sleep($1)' "
+ "AND state = 'active'");
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ pg_fatal("Connection to database failed: %s", PQerrorMessage(monitorConn));
+ }
+ if (PQntuples(res) != 1)
+ {
+ pg_fatal("unexpected number of rows received: %d", PQntuples(res));
+ }
+ if (PQnfields(res) != 1)
+ {
+ pg_fatal("unexpected number of columns received: %d", PQnfields(res));
+ }
+ value = PQgetvalue(res, 0, 0);
+ if (*value != '0')
+ {
+ PQclear(res);
+ break;
+ }
+ PQclear(res);
+
+ /*
+ * wait 10ms before polling again
+ */
+ pg_usleep(10000);
+ }
+}
+
+static void
+test_cancel(PGconn *conn, const char *conninfo)
+{
+ PGcancel *cancel = NULL;
+ PGcancelConn *cancelConn = NULL;
+ PGconn *monitorConn = NULL;
+ char errorbuf[256];
+
+ fprintf(stderr, "test cancellations... ");
+
+ if (PQsetnonblocking(conn, 1) != 0)
+ pg_fatal("failed to set nonblocking mode: %s", PQerrorMessage(conn));
+
+ /*
+ * Make a connection to the database to monitor the query on the main
+ * connection.
+ */
+ monitorConn = PQconnectdb(conninfo);
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ pg_fatal("Connection to database failed: %s",
+ PQerrorMessage(conn));
+ }
+
+ /* test PQcancel */
+ send_cancellable_query(conn, monitorConn);
+ cancel = PQgetCancel(conn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ /* PGcancel object can be reused for the next query */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQcancel(cancel, errorbuf, sizeof(errorbuf)))
+ {
+ pg_fatal("failed to run PQcancel: %s", errorbuf);
+ };
+ confirm_query_canceled(conn);
+
+ PQfreeCancel(cancel);
+
+ /* test PQrequestCancel */
+ send_cancellable_query(conn, monitorConn);
+ if (!PQrequestCancel(conn))
+ pg_fatal("failed to run PQrequestCancel: %s", PQerrorMessage(conn));
+ confirm_query_canceled(conn);
+
+ /* test PQcancelSend */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (!PQcancelSend(cancelConn))
+ pg_fatal("failed to run PQcancelSend: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+ PQcancelFinish(cancelConn);
+
+ /* test PQcancelConn and then polling with PQcancelPoll */
+ send_cancellable_query(conn, monitorConn);
+ cancelConn = PQcancelConn(conn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ /*
+ * test PQcancelReset works on the cancel connection and it can be reused
+ * after
+ */
+ PQcancelReset(cancelConn);
+
+ send_cancellable_query(conn, monitorConn);
+ if (PQcancelStatus(cancelConn) == CONNECTION_BAD)
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ while (true)
+ {
+ struct timeval tv;
+ fd_set input_mask;
+ fd_set output_mask;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancelConn);
+ int sock = PQcancelSocket(cancelConn);
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ FD_ZERO(&input_mask);
+ FD_ZERO(&output_mask);
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ pg_debug("polling for reads\n");
+ FD_SET(sock, &input_mask);
+ break;
+ case PGRES_POLLING_WRITING:
+ pg_debug("polling for writes\n");
+ FD_SET(sock, &output_mask);
+ break;
+ default:
+ pg_fatal("bad cancel connection: %s", PQcancelErrorMessage(cancelConn));
+ }
+
+ if (sock < 0)
+ pg_fatal("sock did not exist: %s", PQcancelErrorMessage(cancelConn));
+
+ tv.tv_sec = 3;
+ tv.tv_usec = 0;
+
+ while (true)
+ {
+ if (select(sock + 1, &input_mask, &output_mask, NULL, &tv) < 0)
+ {
+ if (errno == EINTR)
+ continue;
+ pg_fatal("select() failed: %m");
+ }
+ break;
+ }
+ }
+ if (PQcancelStatus(cancelConn) != CONNECTION_OK)
+ pg_fatal("unexpected cancel connection status: %s", PQcancelErrorMessage(cancelConn));
+ confirm_query_canceled(conn);
+
+ PQcancelFinish(cancelConn);
+
+ fprintf(stderr, "ok\n");
+}
+
static void
test_disallowed_in_pipeline(PGconn *conn)
{
@@ -1789,6 +2047,7 @@ usage(const char *progname)
static void
print_test_list(void)
{
+ printf("cancel\n");
printf("disallowed_in_pipeline\n");
printf("multi_pipelines\n");
printf("nosync\n");
@@ -1890,7 +2149,9 @@ main(int argc, char **argv)
PQTRACE_SUPPRESS_TIMESTAMPS | PQTRACE_REGRESS_MODE);
}
- if (strcmp(testname, "disallowed_in_pipeline") == 0)
+ if (strcmp(testname, "cancel") == 0)
+ test_cancel(conn, conninfo);
+ else if (strcmp(testname, "disallowed_in_pipeline") == 0)
test_disallowed_in_pipeline(conn);
else if (strcmp(testname, "multi_pipelines") == 0)
test_multi_pipelines(conn);
--
2.34.1
[application/octet-stream] v29-0002-libpq-Add-pqReleaseConnHosts-function.patch (2.5K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/3-v29-0002-libpq-Add-pqReleaseConnHosts-function.patch)
download | inline diff:
From b9db005e37e3dce8aa05d4f09e03a1d806bd8bcd Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 17:01:28 +0100
Subject: [PATCH v29 2/5] libpq: Add pqReleaseConnHosts function
In a follow up PR we'll need to free this connhost field in a function
defined in fe-cancel.c
So this extracts the logic to a dedicated extern function.
---
src/interfaces/libpq/fe-connect.c | 38 ++++++++++++++++++++-----------
src/interfaces/libpq/libpq-int.h | 1 +
2 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index c0dea144a00..079abfca9e2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4349,19 +4349,7 @@ freePGconn(PGconn *conn)
free(conn->events[i].name);
}
- /* clean up pg_conn_host structures */
- for (int i = 0; i < conn->nconnhost; ++i)
- {
- free(conn->connhost[i].host);
- free(conn->connhost[i].hostaddr);
- free(conn->connhost[i].port);
- if (conn->connhost[i].password != NULL)
- {
- explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
- free(conn->connhost[i].password);
- }
- }
- free(conn->connhost);
+ pqReleaseConnHosts(conn);
free(conn->client_encoding_initial);
free(conn->events);
@@ -4480,6 +4468,30 @@ release_conn_addrinfo(PGconn *conn)
}
}
+/*
+ * pqReleaseConnHosts
+ * - Free the host list in the PGconn.
+ */
+void
+pqReleaseConnHosts(PGconn *conn)
+{
+ if (conn->connhost)
+ {
+ for (int i = 0; i < conn->nconnhost; ++i)
+ {
+ free(conn->connhost[i].host);
+ free(conn->connhost[i].hostaddr);
+ free(conn->connhost[i].port);
+ if (conn->connhost[i].password != NULL)
+ {
+ explicit_bzero(conn->connhost[i].password, strlen(conn->connhost[i].password));
+ free(conn->connhost[i].password);
+ }
+ }
+ free(conn->connhost);
+ }
+}
+
/*
* sendTerminateConn
* - Send a terminate message to backend.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ff8e0dce776..0d06e260262 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -683,6 +683,7 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
+extern void pqReleaseConnHosts(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
base-commit: 6a1ea02c491d16474a6214603dce40b5b122d4d1
--
2.34.1
[application/octet-stream] v29-0005-Start-using-new-libpq-cancel-APIs.patch (10.5K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/4-v29-0005-Start-using-new-libpq-cancel-APIs.patch)
download | inline diff:
From bb3c4589264f962e0b833450958e49be4fb1f4a8 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Thu, 14 Dec 2023 13:39:09 +0100
Subject: [PATCH v29 5/5] Start using new libpq cancel APIs
A previous commit introduced new APIs to libpq for cancelling queries.
This replaces the usage of the old APIs in the codebase with these newer
ones.
---
contrib/dblink/dblink.c | 30 +++--
contrib/postgres_fdw/connection.c | 105 +++++++++++++++---
.../postgres_fdw/expected/postgres_fdw.out | 15 +++
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 ++
src/fe_utils/connect_utils.c | 11 +-
src/test/isolation/isolationtester.c | 29 ++---
6 files changed, 145 insertions(+), 52 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 19a362526d2..81749b2cdd0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1346,22 +1346,32 @@ PG_FUNCTION_INFO_V1(dblink_cancel_query);
Datum
dblink_cancel_query(PG_FUNCTION_ARGS)
{
- int res;
PGconn *conn;
- PGcancel *cancel;
- char errbuf[256];
+ PGcancelConn *cancelConn;
+ char *msg;
dblink_init();
conn = dblink_get_named_conn(text_to_cstring(PG_GETARG_TEXT_PP(0)));
- cancel = PQgetCancel(conn);
+ cancelConn = PQcancelConn(conn);
- res = PQcancel(cancel, errbuf, 256);
- PQfreeCancel(cancel);
+ PG_TRY();
+ {
+ if (!PQcancelSend(cancelConn))
+ {
+ msg = pchomp(PQcancelErrorMessage(cancelConn));
+ }
+ else
+ {
+ msg = "OK";
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancelConn);
+ }
+ PG_END_TRY();
- if (res == 1)
- PG_RETURN_TEXT_P(cstring_to_text("OK"));
- else
- PG_RETURN_TEXT_P(cstring_to_text(errbuf));
+ PG_RETURN_TEXT_P(cstring_to_text(msg));
}
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4931ebf5915..3ac74ff6a7f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -133,7 +133,7 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
-static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime);
static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
@@ -1315,36 +1315,104 @@ pgfdw_cancel_query(PGconn *conn)
endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
CONNECTION_CLEANUP_TIMEOUT);
- if (!pgfdw_cancel_query_begin(conn))
+ if (!pgfdw_cancel_query_begin(conn, endtime))
return false;
return pgfdw_cancel_query_end(conn, endtime, false);
}
static bool
-pgfdw_cancel_query_begin(PGconn *conn)
+pgfdw_cancel_query_begin(PGconn *conn, TimestampTz endtime)
{
- PGcancel *cancel;
- char errbuf[256];
+ bool timed_out = false;
+ bool failed = false;
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- /*
- * Issue cancel request. Unfortunately, there's no good way to limit the
- * amount of time that we might block inside PQgetCancel().
- */
- if ((cancel = PQgetCancel(conn)))
+
+ if (PQcancelStatus(cancel_conn) == CONNECTION_BAD)
{
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ PG_TRY();
{
ereport(WARNING,
(errcode(ERRCODE_CONNECTION_FAILURE),
errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- return false;
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
}
- PQfreeCancel(cancel);
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+ return false;
}
- return true;
+ /* In what follows, do not leak any PGcancelConn on an error. */
+ PG_TRY();
+ {
+ while (true)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+ long cur_timeout;
+ PostgresPollingStatusType pollres = PQcancelPoll(cancel_conn);
+ int waitEvents = WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH;
+
+ if (pollres == PGRES_POLLING_OK)
+ {
+ break;
+ }
+
+ /* If timeout has expired, give up, else get sleep time. */
+ cur_timeout = TimestampDifferenceMilliseconds(now, endtime);
+ if (cur_timeout <= 0)
+ {
+ timed_out = true;
+ failed = true;
+ goto exit;
+ }
+
+ switch (pollres)
+ {
+ case PGRES_POLLING_READING:
+ waitEvents |= WL_SOCKET_READABLE;
+ break;
+ case PGRES_POLLING_WRITING:
+ waitEvents |= WL_SOCKET_WRITEABLE;
+ break;
+ default:
+ failed = true;
+ goto exit;
+ }
+
+ /* Sleep until there's something to do */
+ WaitLatchOrSocket(MyLatch, waitEvents, PQcancelSocket(cancel_conn),
+ cur_timeout, PG_WAIT_EXTENSION);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+exit: ;
+ if (failed)
+ {
+ if (timed_out)
+ {
+ ereport(WARNING,
+ (errmsg("could not cancel request due to timeout")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ pchomp(PQcancelErrorMessage(cancel_conn)))));
+ }
+ }
+ }
+ PG_FINALLY();
+ {
+ PQcancelFinish(cancel_conn);
+ }
+ PG_END_TRY();
+
+ return !failed;
}
static bool
@@ -1685,7 +1753,10 @@ pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
- if (!pgfdw_cancel_query_begin(entry->conn))
+ TimestampTz endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(entry->conn, endtime))
return false; /* Unable to cancel running query */
*cancel_requested = lappend(*cancel_requested, entry);
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b5a38aeb214..16206a23a9d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2698,6 +2698,21 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
(10 rows)
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Foreign Scan
+ Output: (count(*))
+ Relations: Aggregate on ((((public.ft1) INNER JOIN (public.ft2)) INNER JOIN (public.ft4)) INNER JOIN (public.ft5))
+ Remote SQL: SELECT count(*) FROM ((("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) INNER JOIN "S 1"."T 3" r4 ON (TRUE)) INNER JOIN "S 1"."T 4" r6 ON (TRUE))
+(4 rows)
+
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+ERROR: canceling statement due to statement timeout
+RESET statement_timeout;
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index f410c3db4e6..01a98750611 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -717,6 +717,13 @@ SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c
SELECT t1.c1, t2.c2 FROM v4 t1 LEFT JOIN ft5 t2 ON (t1.c1 = t2.c1) ORDER BY t1.c1, t2.c1 OFFSET 10 LIMIT 10;
ALTER VIEW v4 OWNER TO regress_view_owner;
+-- Make sure this big CROSS JOIN query is pushed down
+EXPLAIN (VERBOSE, COSTS OFF) SELECT count(*) FROM ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5;
+-- Make sure query cancellation works
+SET statement_timeout = '10ms';
+select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long
+RESET statement_timeout;
+
-- ====================================================================
-- Check that userid to use when querying the remote table is correctly
-- propagated into foreign rels present in subqueries under an UNION ALL
diff --git a/src/fe_utils/connect_utils.c b/src/fe_utils/connect_utils.c
index 808d54461fd..c5cd2f57875 100644
--- a/src/fe_utils/connect_utils.c
+++ b/src/fe_utils/connect_utils.c
@@ -157,19 +157,14 @@ connectMaintenanceDatabase(ConnParams *cparams,
void
disconnectDatabase(PGconn *conn)
{
- char errbuf[256];
-
Assert(conn != NULL);
if (PQtransactionStatus(conn) == PQTRANS_ACTIVE)
{
- PGcancel *cancel;
+ PGcancelConn *cancelConn = PQcancelConn(conn);
- if ((cancel = PQgetCancel(conn)))
- {
- (void) PQcancel(cancel, errbuf, sizeof(errbuf));
- PQfreeCancel(cancel);
- }
+ (void) PQcancelSend(cancelConn);
+ PQcancelFinish(cancelConn);
}
PQfinish(conn);
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index 0a66235153a..de31a875716 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -946,26 +946,21 @@ try_complete_step(TestSpec *testspec, PermutationStep *pstep, int flags)
*/
if (td > max_step_wait && !canceled)
{
- PGcancel *cancel = PQgetCancel(conn);
+ PGcancelConn *cancel_conn = PQcancelConn(conn);
- if (cancel != NULL)
+ if (PQcancelSend(cancel_conn))
{
- char buf[256];
-
- if (PQcancel(cancel, buf, sizeof(buf)))
- {
- /*
- * print to stdout not stderr, as this should appear
- * in the test case's results
- */
- printf("isolationtester: canceling step %s after %d seconds\n",
- step->name, (int) (td / USECS_PER_SEC));
- canceled = true;
- }
- else
- fprintf(stderr, "PQcancel failed: %s\n", buf);
- PQfreeCancel(cancel);
+ /*
+ * print to stdout not stderr, as this should appear in
+ * the test case's results
+ */
+ printf("isolationtester: canceling step %s after %d seconds\n",
+ step->name, (int) (td / USECS_PER_SEC));
+ canceled = true;
}
+ else
+ fprintf(stderr, "PQcancel failed: %s\n", PQcancelErrorMessage(cancel_conn));
+ PQcancelFinish(cancel_conn);
}
/*
--
2.34.1
[application/octet-stream] v29-0003-libpq-Change-some-static-functions-to-extern.patch (9.7K, ../../CAGECzQRLpsk31bKa-iq6+nY0QYpYg_or7UK9n-z-DH0B07haOg@mail.gmail.com/5-v29-0003-libpq-Change-some-static-functions-to-extern.patch)
download | inline diff:
From 01991be38133f43dd89328ca74f0653d1bb4ca27 Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <[email protected]>
Date: Fri, 26 Jan 2024 16:47:51 +0100
Subject: [PATCH v29 3/5] libpq: Change some static functions to extern
This is in preparation of a follow up commit that starts using these
functions from fe-cancel.c.
---
src/interfaces/libpq/fe-connect.c | 85 +++++++++++++++----------------
src/interfaces/libpq/libpq-int.h | 6 +++
2 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 079abfca9e2..dd240d42d70 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -387,15 +387,10 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
-static bool connectOptions2(PGconn *conn);
-static int connectDBStart(PGconn *conn);
-static int connectDBComplete(PGconn *conn);
static PGPing internal_ping(PGconn *conn);
-static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
-static void closePGconn(PGconn *conn);
static void release_conn_addrinfo(PGconn *conn);
static int store_conn_addrinfo(PGconn *conn, struct addrinfo *addrlist);
static void sendTerminateConn(PGconn *conn);
@@ -644,7 +639,7 @@ pqDropServerData(PGconn *conn)
* PQconnectStart or PQconnectStartParams (which differ in the same way as
* PQconnectdb and PQconnectdbParams) and PQconnectPoll.
*
- * Internally, the static functions connectDBStart, connectDBComplete
+ * Internally, the static functions pqConnectDBStart, pqConnectDBComplete
* are part of the connection procedure.
*/
@@ -678,7 +673,7 @@ PQconnectdbParams(const char *const *keywords,
PGconn *conn = PQconnectStartParams(keywords, values, expand_dbname);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -731,7 +726,7 @@ PQconnectdb(const char *conninfo)
PGconn *conn = PQconnectStart(conninfo);
if (conn && conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
return conn;
}
@@ -785,7 +780,7 @@ PQconnectStartParams(const char *const *keywords,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -819,15 +814,15 @@ PQconnectStartParams(const char *const *keywords,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -863,7 +858,7 @@ PQconnectStart(const char *conninfo)
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -876,15 +871,15 @@ PQconnectStart(const char *conninfo)
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (!connectDBStart(conn))
+ if (!pqConnectDBStart(conn))
{
- /* Just in case we failed to set it in connectDBStart */
+ /* Just in case we failed to set it in pqConnectDBStart */
conn->status = CONNECTION_BAD;
}
@@ -895,7 +890,7 @@ PQconnectStart(const char *conninfo)
* Move option values into conn structure
*
* Don't put anything cute here --- intelligence should be in
- * connectOptions2 ...
+ * pqConnectOptions2 ...
*
* Returns true on success. On failure, returns false and sets error message.
*/
@@ -933,7 +928,7 @@ fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
*
* Internal subroutine to set up connection parameters given an already-
* created PGconn and a conninfo string. Derived settings should be
- * processed by calling connectOptions2 next. (We split them because
+ * processed by calling pqConnectOptions2 next. (We split them because
* PQsetdbLogin overrides defaults in between.)
*
* Returns true if OK, false if trouble (in which case errorMessage is set
@@ -1055,15 +1050,15 @@ libpq_prng_init(PGconn *conn)
}
/*
- * connectOptions2
+ * pqConnectOptions2
*
* Compute derived connection options after absorbing all user-supplied info.
*
* Returns true if OK, false if trouble (in which case errorMessage is set
* and so is conn->status).
*/
-static bool
-connectOptions2(PGconn *conn)
+bool
+pqConnectOptions2(PGconn *conn)
{
int i;
@@ -1822,7 +1817,7 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
* to initialize conn->errorMessage to empty. All subsequent steps during
* connection initialization will only append to that buffer.
*/
- conn = makeEmptyPGconn();
+ conn = pqMakeEmptyPGconn();
if (conn == NULL)
return NULL;
@@ -1901,14 +1896,14 @@ PQsetdbLogin(const char *pghost, const char *pgport, const char *pgoptions,
/*
* Compute derived options
*/
- if (!connectOptions2(conn))
+ if (!pqConnectOptions2(conn))
return conn;
/*
* Connect to the database
*/
- if (connectDBStart(conn))
- (void) connectDBComplete(conn);
+ if (pqConnectDBStart(conn))
+ (void) pqConnectDBComplete(conn);
return conn;
@@ -2277,14 +2272,14 @@ setTCPUserTimeout(PGconn *conn)
}
/* ----------
- * connectDBStart -
+ * pqConnectDBStart -
* Begin the process of making a connection to the backend.
*
* Returns 1 if successful, 0 if not.
* ----------
*/
-static int
-connectDBStart(PGconn *conn)
+int
+pqConnectDBStart(PGconn *conn)
{
if (!conn)
return 0;
@@ -2347,14 +2342,14 @@ connect_errReturn:
/*
- * connectDBComplete
+ * pqConnectDBComplete
*
* Block and complete a connection.
*
* Returns 1 on success, 0 on failure.
*/
-static int
-connectDBComplete(PGconn *conn)
+int
+pqConnectDBComplete(PGconn *conn)
{
PostgresPollingStatusType flag = PGRES_POLLING_WRITING;
time_t finish_time = ((time_t) -1);
@@ -2704,7 +2699,7 @@ keep_going: /* We will come back to here until there is
* combining it with the insertion.
*
* We don't need to initialize conn->prng_state here, because that
- * already happened in connectOptions2.
+ * already happened in pqConnectOptions2.
*/
for (int i = 1; i < conn->naddr; i++)
{
@@ -4181,7 +4176,7 @@ internal_ping(PGconn *conn)
/* Attempt to complete the connection */
if (conn->status != CONNECTION_BAD)
- (void) connectDBComplete(conn);
+ (void) pqConnectDBComplete(conn);
/* Definitely OK if we succeeded */
if (conn->status != CONNECTION_BAD)
@@ -4233,11 +4228,11 @@ internal_ping(PGconn *conn)
/*
- * makeEmptyPGconn
+ * pqMakeEmptyPGconn
* - create a PGconn data structure with (as yet) no interesting data
*/
-static PGconn *
-makeEmptyPGconn(void)
+PGconn *
+pqMakeEmptyPGconn(void)
{
PGconn *conn;
@@ -4330,7 +4325,7 @@ makeEmptyPGconn(void)
* freePGconn
* - free an idle (closed) PGconn data structure
*
- * NOTE: this should not overlap any functionality with closePGconn().
+ * NOTE: this should not overlap any functionality with pqClosePGconn().
* Clearing/resetting of transient state belongs there; what we do here is
* release data that is to be held for the life of the PGconn structure.
* If a value ought to be cleared/freed during PQreset(), do it there not here.
@@ -4516,15 +4511,15 @@ sendTerminateConn(PGconn *conn)
}
/*
- * closePGconn
+ * pqClosePGconn
* - properly close a connection to the backend
*
* This should reset or release all transient state, but NOT the connection
* parameters. On exit, the PGconn should be in condition to start a fresh
* connection with the same parameters (see PQreset()).
*/
-static void
-closePGconn(PGconn *conn)
+void
+pqClosePGconn(PGconn *conn)
{
/*
* If possible, send Terminate message to close the connection politely.
@@ -4567,7 +4562,7 @@ PQfinish(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
freePGconn(conn);
}
}
@@ -4581,9 +4576,9 @@ PQreset(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- if (connectDBStart(conn) && connectDBComplete(conn))
+ if (pqConnectDBStart(conn) && pqConnectDBComplete(conn))
{
/*
* Notify event procs of successful reset.
@@ -4614,9 +4609,9 @@ PQresetStart(PGconn *conn)
{
if (conn)
{
- closePGconn(conn);
+ pqClosePGconn(conn);
- return connectDBStart(conn);
+ return pqConnectDBStart(conn);
}
return 0;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 0d06e260262..07732927a5b 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -684,6 +684,12 @@ extern bool pqGetHomeDirectory(char *buf, int bufsize);
extern bool pqParseIntParam(const char *value, int *result, PGconn *conn,
const char *context);
extern void pqReleaseConnHosts(PGconn *conn);
+extern bool pqConnectOptions2(PGconn *conn);
+extern int pqConnectDBStart(PGconn *conn);
+extern int pqConnectDBComplete(PGconn *conn);
+extern PGconn *pqMakeEmptyPGconn(void);
+extern bool pqCopyPGconn(PGconn *srcConn, PGconn *dstConn);
+extern void pqClosePGconn(PGconn *conn);
extern pgthreadlock_t pg_g_threadlock;
--
2.34.1
^ permalink raw reply [nested|flat] 36+ messages in thread
* [PATCH v16 2/8] Row pattern recognition patch (parse/analysis).
@ 2024-04-12 06:49 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 36+ messages in thread
From: Tatsuo Ishii @ 2024-04-12 06:49 UTC (permalink / raw)
---
src/backend/parser/parse_agg.c | 7 +
src/backend/parser/parse_clause.c | 296 +++++++++++++++++++++++++++++-
src/backend/parser/parse_expr.c | 4 +
src/backend/parser/parse_func.c | 3 +
4 files changed, 309 insertions(+), 1 deletion(-)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index bee7d8346a..9bc22a836a 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -577,6 +577,10 @@ check_agglevels_and_constraints(ParseState *pstate, Node *expr)
errkind = true;
break;
+ case EXPR_KIND_RPR_DEFINE:
+ errkind = true;
+ break;
+
/*
* There is intentionally no default: case here, so that the
* compiler will warn if we add a new ParseExprKind without
@@ -967,6 +971,9 @@ transformWindowFuncCall(ParseState *pstate, WindowFunc *wfunc,
case EXPR_KIND_CYCLE_MARK:
errkind = true;
break;
+ case EXPR_KIND_RPR_DEFINE:
+ errkind = true;
+ break;
/*
* There is intentionally no default: case here, so that the
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4fc5fc87e0..003a1e14ce 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -98,7 +98,14 @@ static WindowClause *findWindowClause(List *wclist, const char *name);
static Node *transformFrameOffset(ParseState *pstate, int frameOptions,
Oid rangeopfamily, Oid rangeopcintype, Oid *inRangeFunc,
Node *clause);
-
+static void transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef,
+ List **targetlist);
+static List *transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef,
+ List **targetlist);
+static void transformPatternClause(ParseState *pstate, WindowClause *wc,
+ WindowDef *windef);
+static List *transformMeasureClause(ParseState *pstate, WindowClause *wc,
+ WindowDef *windef);
/*
* transformFromClause -
@@ -2956,6 +2963,10 @@ transformWindowDefinitions(ParseState *pstate,
rangeopfamily, rangeopcintype,
&wc->endInRangeFunc,
windef->endOffset);
+
+ /* Process Row Pattern Recognition related clauses */
+ transformRPR(pstate, wc, windef, targetlist);
+
wc->runCondition = NIL;
wc->winref = winref;
@@ -3821,3 +3832,286 @@ transformFrameOffset(ParseState *pstate, int frameOptions,
return node;
}
+
+/*
+ * transformRPR
+ * Process Row Pattern Recognition related clauses
+ */
+static void
+transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef,
+ List **targetlist)
+{
+ /*
+ * Window definition exists?
+ */
+ if (windef == NULL)
+ return;
+
+ /*
+ * Row Pattern Common Syntax clause exists?
+ */
+ if (windef->rpCommonSyntax == NULL)
+ return;
+
+ /* Check Frame option. Frame must start at current row */
+ if ((wc->frameOptions & FRAMEOPTION_START_CURRENT_ROW) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("FRAME must start at current row when row patttern recognition is used")));
+
+ /* Transform AFTER MACH SKIP TO clause */
+ wc->rpSkipTo = windef->rpCommonSyntax->rpSkipTo;
+
+ /* Transform AFTER MACH SKIP TO variable */
+ wc->rpSkipVariable = windef->rpCommonSyntax->rpSkipVariable;
+
+ /* Transform SEEK or INITIAL clause */
+ wc->initial = windef->rpCommonSyntax->initial;
+
+ /* Transform DEFINE clause into list of TargetEntry's */
+ wc->defineClause = transformDefineClause(pstate, wc, windef, targetlist);
+
+ /* Check PATTERN clause and copy to patternClause */
+ transformPatternClause(pstate, wc, windef);
+
+ /* Transform MEASURE clause */
+ transformMeasureClause(pstate, wc, windef);
+}
+
+/*
+ * transformDefineClause Process DEFINE clause and transform ResTarget into
+ * list of TargetEntry.
+ *
+ * XXX we only support column reference in row pattern definition search
+ * condition, e.g. "price". <row pattern definition variable name>.<column
+ * reference> is not supported, e.g. "A.price".
+ */
+static List *
+transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef,
+ List **targetlist)
+{
+ /* DEFINE variable name initials */
+ static char *defineVariableInitials = "abcdefghijklmnopqrstuvwxyz";
+
+ ListCell *lc,
+ *l;
+ ResTarget *restarget,
+ *r;
+ List *restargets;
+ List *defineClause;
+ char *name;
+ int initialLen;
+ int i;
+
+ /*
+ * If Row Definition Common Syntax exists, DEFINE clause must exist. (the
+ * raw parser should have already checked it.)
+ */
+ Assert(windef->rpCommonSyntax->rpDefs != NULL);
+
+ /*
+ * Check and add "A AS A IS TRUE" if pattern variable is missing in DEFINE
+ * per the SQL standard.
+ */
+ restargets = NIL;
+ foreach(lc, windef->rpCommonSyntax->rpPatterns)
+ {
+ A_Expr *a;
+ bool found = false;
+
+ if (!IsA(lfirst(lc), A_Expr))
+ ereport(ERROR,
+ errmsg("node type is not A_Expr"));
+
+ a = (A_Expr *) lfirst(lc);
+ name = strVal(a->lexpr);
+
+ foreach(l, windef->rpCommonSyntax->rpDefs)
+ {
+ restarget = (ResTarget *) lfirst(l);
+
+ if (!strcmp(restarget->name, name))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found)
+ {
+ /*
+ * "name" is missing. So create "name AS name IS TRUE" ResTarget
+ * node and add it to the temporary list.
+ */
+ A_Const *n;
+
+ restarget = makeNode(ResTarget);
+ n = makeNode(A_Const);
+ n->val.boolval.type = T_Boolean;
+ n->val.boolval.boolval = true;
+ n->location = -1;
+ restarget->name = pstrdup(name);
+ restarget->indirection = NIL;
+ restarget->val = (Node *) n;
+ restarget->location = -1;
+ restargets = lappend((List *) restargets, restarget);
+ }
+ }
+
+ if (list_length(restargets) >= 1)
+ {
+ /* add missing DEFINEs */
+ windef->rpCommonSyntax->rpDefs =
+ list_concat(windef->rpCommonSyntax->rpDefs, restargets);
+ list_free(restargets);
+ }
+
+ /*
+ * Check for duplicate row pattern definition variables. The standard
+ * requires that no two row pattern definition variable names shall be
+ * equivalent.
+ */
+ restargets = NIL;
+ foreach(lc, windef->rpCommonSyntax->rpDefs)
+ {
+ restarget = (ResTarget *) lfirst(lc);
+ name = restarget->name;
+
+ /*
+ * Add DEFINE expression (Restarget->val) to the targetlist as a
+ * TargetEntry if it does not exist yet. Planner will add the column
+ * ref var node to the outer plan's target list later on. This makes
+ * DEFINE expression could access the outer tuple while evaluating
+ * PATTERN.
+ *
+ * XXX: adding whole expressions of DEFINE to the plan.targetlist is
+ * not so good, because it's not necessary to evalute the expression
+ * in the target list while running the plan. We should extract the
+ * var nodes only then add them to the plan.targetlist.
+ */
+ findTargetlistEntrySQL99(pstate, (Node *) restarget->val,
+ targetlist, EXPR_KIND_RPR_DEFINE);
+
+ /*
+ * Make sure that the row pattern definition search condition is a
+ * boolean expression.
+ */
+ transformWhereClause(pstate, restarget->val,
+ EXPR_KIND_RPR_DEFINE, "DEFINE");
+
+ foreach(l, restargets)
+ {
+ char *n;
+
+ r = (ResTarget *) lfirst(l);
+ n = r->name;
+
+ if (!strcmp(n, name))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("row pattern definition variable name \"%s\" appears more than once in DEFINE clause",
+ name),
+ parser_errposition(pstate, exprLocation((Node *) r))));
+ }
+ restargets = lappend(restargets, restarget);
+ }
+ list_free(restargets);
+
+ /*
+ * Create list of row pattern DEFINE variable name's initial. We assign
+ * [a-z] to them (up to 26 variable names are allowed).
+ */
+ restargets = NIL;
+ i = 0;
+ initialLen = strlen(defineVariableInitials);
+
+ foreach(lc, windef->rpCommonSyntax->rpDefs)
+ {
+ char initial[2];
+
+ restarget = (ResTarget *) lfirst(lc);
+ name = restarget->name;
+
+ if (i >= initialLen)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("number of row pattern definition variable names exceeds %d",
+ initialLen),
+ parser_errposition(pstate,
+ exprLocation((Node *) restarget))));
+ }
+ initial[0] = defineVariableInitials[i++];
+ initial[1] = '\0';
+ wc->defineInitial = lappend(wc->defineInitial,
+ makeString(pstrdup(initial)));
+ }
+
+ defineClause = transformTargetList(pstate, windef->rpCommonSyntax->rpDefs,
+ EXPR_KIND_RPR_DEFINE);
+
+ /* mark column origins */
+ markTargetListOrigins(pstate, defineClause);
+
+ /* mark all nodes in the DEFINE clause tree with collation information */
+ assign_expr_collations(pstate, (Node *) defineClause);
+
+ return defineClause;
+}
+
+/*
+ * transformPatternClause
+ * Process PATTERN clause and return PATTERN clause in the raw parse tree
+ */
+static void
+transformPatternClause(ParseState *pstate, WindowClause *wc,
+ WindowDef *windef)
+{
+ ListCell *lc;
+
+ /*
+ * Row Pattern Common Syntax clause exists?
+ */
+ if (windef->rpCommonSyntax == NULL)
+ return;
+
+ wc->patternVariable = NIL;
+ wc->patternRegexp = NIL;
+ foreach(lc, windef->rpCommonSyntax->rpPatterns)
+ {
+ A_Expr *a;
+ char *name;
+ char *regexp;
+
+ if (!IsA(lfirst(lc), A_Expr))
+ ereport(ERROR,
+ errmsg("node type is not A_Expr"));
+
+ a = (A_Expr *) lfirst(lc);
+ name = strVal(a->lexpr);
+
+ wc->patternVariable = lappend(wc->patternVariable, makeString(pstrdup(name)));
+ regexp = strVal(lfirst(list_head(a->name)));
+
+ wc->patternRegexp = lappend(wc->patternRegexp, makeString(pstrdup(regexp)));
+ }
+}
+
+/*
+ * transformMeasureClause
+ * Process MEASURE clause
+ * XXX MEASURE clause is not supported yet
+ */
+static List *
+transformMeasureClause(ParseState *pstate, WindowClause *wc,
+ WindowDef *windef)
+{
+ if (windef->rowPatternMeasures == NIL)
+ return NIL;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("%s", "MEASURE clause is not supported yet"),
+ parser_errposition(pstate, exprLocation((Node *) windef->rowPatternMeasures))));
+ return NIL;
+}
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 4c98d7a046..a9f9d47854 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -578,6 +578,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref)
case EXPR_KIND_COPY_WHERE:
case EXPR_KIND_GENERATED_COLUMN:
case EXPR_KIND_CYCLE_MARK:
+ case EXPR_KIND_RPR_DEFINE:
/* okay */
break;
@@ -1817,6 +1818,7 @@ transformSubLink(ParseState *pstate, SubLink *sublink)
case EXPR_KIND_VALUES:
case EXPR_KIND_VALUES_SINGLE:
case EXPR_KIND_CYCLE_MARK:
+ case EXPR_KIND_RPR_DEFINE:
/* okay */
break;
case EXPR_KIND_CHECK_CONSTRAINT:
@@ -3197,6 +3199,8 @@ ParseExprKindName(ParseExprKind exprKind)
return "GENERATED AS";
case EXPR_KIND_CYCLE_MARK:
return "CYCLE";
+ case EXPR_KIND_RPR_DEFINE:
+ return "DEFINE";
/*
* There is intentionally no default: case here, so that the
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
index 0cbc950c95..ad982a7c17 100644
--- a/src/backend/parser/parse_func.c
+++ b/src/backend/parser/parse_func.c
@@ -2657,6 +2657,9 @@ check_srf_call_placement(ParseState *pstate, Node *last_srf, int location)
case EXPR_KIND_CYCLE_MARK:
errkind = true;
break;
+ case EXPR_KIND_RPR_DEFINE:
+ errkind = true;
+ break;
/*
* There is intentionally no default: case here, so that the
--
2.25.1
----Next_Part(Fri_Apr_12_16_09_08_2024_262)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v16-0003-Row-pattern-recognition-patch-rewriter.patch"
^ permalink raw reply [nested|flat] 36+ messages in thread
end of thread, other threads:[~2024-04-12 06:49 UTC | newest]
Thread overview: 36+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-12-19 20:32 [PATCH v18 16/18] WIP: Move xid horizon computation for page level index vacuum to primary. Andres Freund <[email protected]>
2022-01-12 15:22 Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-01-13 00:44 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-01-13 14:51 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-09 00:27 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-03-24 21:41 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-24 22:49 ` Re: Add non-blocking version of PQcancel Andres Freund <[email protected]>
2022-03-25 18:34 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 18:46 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-25 19:22 ` Re: Add non-blocking version of PQcancel Robert Haas <[email protected]>
2022-03-25 19:34 ` Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-03-28 09:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-30 16:08 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-03-31 05:47 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-25 00:36 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 11:45 ` Re: Add non-blocking version of PQcancel Justin Pryzby <[email protected]>
2022-06-27 12:29 ` Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2022-06-27 09:29 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-09-14 21:53 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Tom Lane <[email protected]>
2022-10-05 13:23 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-04 15:58 ` Re: Add non-blocking version of PQcancel Jacob Champion <[email protected]>
2022-11-15 11:38 ` Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2022-11-29 19:17 ` Re: Add non-blocking version of PQcancel Daniel Gustafsson <[email protected]>
2022-11-30 09:20 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema <[email protected]>
2024-01-26 01:59 Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-26 10:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 12:11 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 16:52 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-26 17:19 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-26 23:14 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 03:15 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel vignesh C <[email protected]>
2024-01-28 09:51 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-28 12:39 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-01-29 11:44 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Alvaro Herrera <[email protected]>
2024-01-29 12:28 ` Re: [EXTERNAL] Re: Add non-blocking version of PQcancel Jelte Fennema-Nio <[email protected]>
2024-04-12 06:49 [PATCH v16 2/8] Row pattern recognition patch (parse/analysis). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox