public inbox for [email protected]
help / color / mirror / Atom feedpostgres_fdw: commit remote (sub)transactions in parallel during pre-commit
22+ messages / 5 participants
[nested] [flat]
* postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
@ 2021-10-31 09:05 Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-01 22:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
0 siblings, 2 replies; 22+ messages in thread
From: Etsuro Fujita @ 2021-10-31 09:05 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>; +Cc: Fujii Masao <[email protected]>
Hi,
As I said before [1], I’m working on $SUBJECT. Attached is a WIP
patch for that. The patch is pretty simple: if a server option added
by the patch “parallel_commit” is enabled, 1) asynchronously send
COMMIT TRANSACTION (RELEASE SAVEPOINT) to all remote servers involved
in a local (sub)transaction, then 2) wait for the results from the
remote servers in the order that the command was sent to the remote
servers, when called from pgfdw_xact_callback (pgfdw_subxact_callback)
during pre-commit. The patch also parallelizes clearing prepared
statements the same way during pre-commit. (The option is false by
default.)
I evaluated the effectiveness of the patch using a simple
multi-statement transaction:
BEGIN;
SAVEPOINT s;
INSERT INTO ft1 VALUES (10, 10);
INSERT INTO ft2 VALUES (20, 20);
RELEASE SAVEPOINT s;
COMMIT;
where ft1 and ft2 are foreign tables created on different foreign
servers hosted on different machines. I ran the transaction five
times using the patch with the option enabled/disabled, and measured
the latencies for the RELEASE and COMMIT commands in each run. The
average latencies for these commands over the five runs are:
* RELEASE
parallel_commit=0: 0.385 ms
parallel_commit=1: 0.221 ms
* COMMIT
parallel_commit=0: 1.660 ms
parallel_commit=1: 0.861 ms
With the option enabled, the average latencies for both commands are
reduced significantly!
I think we could extend this to abort cleanup of remote
(sub)transactions during post-abort. Anyway, I think this is useful,
so I’ll add this to the upcoming commitfest.
Best regards,
Etsuro Fujita
[1] https://www.postgresql.org/message-id/CAPmGK177E6HPcCQB4-s%2Bm9AcCZDHCC2drZy%2BFKnnvEXw9kXoXQ%40mail...
Attachments:
[application/octet-stream] postgres-fdw-parallel-commit-1.patch (14.7K, ../../CAPmGK17dAZCXvwnfpr1eTfknTGdt=hYTV9405Gt5SqPOX8K84w@mail.gmail.com/2-postgres-fdw-parallel-commit-1.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 4aff315b7c..03223afd53 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -58,6 +58,7 @@ typedef struct ConnCacheEntry
bool have_prep_stmt; /* have we prepared any stmts in this xact? */
bool have_error; /* have any subxacts aborted in this xact? */
bool changing_xact_state; /* xact state change in process */
+ bool parallel_commit; /* do we commit (sub)xacts in parallel? */
bool invalidated; /* true if reconnect is pending */
bool keep_connections; /* setting value of keep_connections
* server option */
@@ -92,6 +93,8 @@ static PGconn *connect_pg_server(ForeignServer *server, UserMapping *user);
static void disconnect_pg_server(ConnCacheEntry *entry);
static void check_conn_params(const char **keywords, const char **values, UserMapping *user);
static void configure_remote_session(PGconn *conn);
+static void do_sql_command_begin(PGconn *conn, const char *sql);
+static void do_sql_command_end(PGconn *conn, const char *sql, bool ignore_errors);
static void begin_remote_xact(ConnCacheEntry *entry);
static void pgfdw_xact_callback(XactEvent event, void *arg);
static void pgfdw_subxact_callback(SubXactEvent event,
@@ -100,6 +103,7 @@ static void pgfdw_subxact_callback(SubXactEvent event,
void *arg);
static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
+static void pgfdw_reset_xact_nesting_depth(ConnCacheEntry *entry);
static bool pgfdw_cancel_query(PGconn *conn);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
@@ -318,12 +322,15 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
* By default, all the connections to any foreign servers are kept open.
*/
entry->keep_connections = true;
+ entry->parallel_commit = false;
foreach(lc, server->options)
{
DefElem *def = (DefElem *) lfirst(lc);
if (strcmp(def->defname, "keep_connections") == 0)
entry->keep_connections = defGetBoolean(def);
+ if (strcmp(def->defname, "parallel_commit") == 0)
+ entry->parallel_commit = defGetBoolean(def);
}
/* Now try to make the connection */
@@ -589,11 +596,36 @@ do_sql_command(PGconn *conn, const char *sql)
{
PGresult *res;
+ do_sql_command_begin(conn, sql);
+ res = pgfdw_get_result(conn, sql);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ pgfdw_report_error(ERROR, res, conn, true, sql);
+ PQclear(res);
+}
+
+static void
+do_sql_command_begin(PGconn *conn, const char *sql)
+{
if (!PQsendQuery(conn, sql))
pgfdw_report_error(ERROR, NULL, conn, false, sql);
+}
+
+static void
+do_sql_command_end(PGconn *conn, const char *sql, bool ignore_errors)
+{
+ PGresult *res;
+
+ /* Consume whatever data is available from the socket */
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, sql);
res = pgfdw_get_result(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
- pgfdw_report_error(ERROR, res, conn, true, sql);
+ {
+ if (ignore_errors)
+ pgfdw_report_error(WARNING, res, conn, true, sql);
+ else
+ pgfdw_report_error(ERROR, res, conn, true, sql);
+ }
PQclear(res);
}
@@ -851,6 +883,7 @@ pgfdw_xact_callback(XactEvent event, void *arg)
{
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
+ List *pending_xacts = NIL;
/* Quick exit if no connections were touched in this transaction. */
if (!xact_got_connection)
@@ -888,6 +921,13 @@ pgfdw_xact_callback(XactEvent event, void *arg)
/* Commit all remote transactions during pre-commit */
entry->changing_xact_state = true;
+ if (entry->parallel_commit)
+ {
+ do_sql_command_begin(entry->conn,
+ "COMMIT TRANSACTION");
+ pending_xacts = lappend(pending_xacts, entry);
+ continue;
+ }
do_sql_command(entry->conn, "COMMIT TRANSACTION");
entry->changing_xact_state = false;
@@ -943,23 +983,68 @@ pgfdw_xact_callback(XactEvent event, void *arg)
}
}
- /* Reset state to show we're out of a transaction */
- entry->xact_depth = 0;
-
/*
- * If the connection isn't in a good idle state, it is marked as
- * invalid or keep_connections option of its server is disabled, then
- * discard it to recover. Next GetConnection will open a new
- * connection.
+ * Reset state to show we're out of a transaction, and, if necessary,
+ * discard the connection
*/
- if (PQstatus(entry->conn) != CONNECTION_OK ||
- PQtransactionStatus(entry->conn) != PQTRANS_IDLE ||
- entry->changing_xact_state ||
- entry->invalidated ||
- !entry->keep_connections)
+ pgfdw_reset_xact_nesting_depth(entry);
+ }
+
+ if (pending_xacts)
+ {
+ List *pending_deallocs = NIL;
+ ListCell *lc;
+
+ Assert(event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
+ event == XACT_EVENT_PRE_COMMIT);
+
+ foreach(lc, pending_xacts)
+ {
+ entry = (ConnCacheEntry *) lfirst(lc);
+
+ Assert(entry->changing_xact_state);
+ do_sql_command_end(entry->conn, "COMMIT TRANSACTION", false);
+ entry->changing_xact_state = false;
+
+ /* Do a DEALLOCATE ALL if needed */
+ if (entry->have_prep_stmt && entry->have_error)
+ {
+ entry->changing_xact_state = true;
+ do_sql_command_begin(entry->conn, "DEALLOCATE ALL");
+ pending_deallocs = lappend(pending_deallocs, entry);
+ continue;
+ }
+
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /*
+ * Reset state to show we're out of a transaction, and, if
+ * necessary, discard the connection
+ */
+ pgfdw_reset_xact_nesting_depth(entry);
+ }
+
+ if (pending_deallocs)
{
- elog(DEBUG3, "discarding connection %p", entry->conn);
- disconnect_pg_server(entry);
+ foreach(lc, pending_deallocs)
+ {
+ entry = (ConnCacheEntry *) lfirst(lc);
+
+ Assert(entry->changing_xact_state);
+ /* Ignore errors in the DEALLOCATE (see note above) */
+ do_sql_command_end(entry->conn, "DEALLOCATE ALL", true);
+ entry->changing_xact_state = false;
+
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /*
+ * Reset state to show we're out of a transaction, and, if
+ * necessary, discard the connection
+ */
+ pgfdw_reset_xact_nesting_depth(entry);
+ }
}
}
@@ -984,6 +1069,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
int curlevel;
+ List *pending_subxacts = NIL;
/* Nothing to do at subxact start, nor after commit. */
if (!(event == SUBXACT_EVENT_PRE_COMMIT_SUB ||
@@ -1026,6 +1112,12 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
/* Commit all remote subtransactions during pre-commit */
snprintf(sql, sizeof(sql), "RELEASE SAVEPOINT s%d", curlevel);
entry->changing_xact_state = true;
+ if (entry->parallel_commit)
+ {
+ do_sql_command_begin(entry->conn, sql);
+ pending_subxacts = lappend(pending_subxacts, entry);
+ continue;
+ }
do_sql_command(entry->conn, sql);
entry->changing_xact_state = false;
}
@@ -1041,6 +1133,27 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
/* OK, we're outta that level of subtransaction */
entry->xact_depth--;
}
+
+ if (pending_subxacts)
+ {
+ char sql[100];
+ ListCell *lc;
+
+ Assert(event == SUBXACT_EVENT_PRE_COMMIT_SUB);
+
+ snprintf(sql, sizeof(sql), "RELEASE SAVEPOINT s%d", curlevel);
+ foreach(lc, pending_subxacts)
+ {
+ entry = (ConnCacheEntry *) lfirst(lc);
+
+ Assert(entry->changing_xact_state);
+ do_sql_command_end(entry->conn, sql, false);
+ entry->changing_xact_state = false;
+
+ /* OK, we're outta that level of subtransaction */
+ entry->xact_depth--;
+ }
+ }
}
/*
@@ -1132,6 +1245,28 @@ pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry)
server->servername)));
}
+static void
+pgfdw_reset_xact_nesting_depth(ConnCacheEntry *entry)
+{
+ /* Reset state to show we're out of a transaction */
+ entry->xact_depth = 0;
+
+ /*
+ * If the connection isn't in a good idle state, it is marked as invalid
+ * or keep_connections option of its server is disabled, then discard it
+ * to recover. Next GetConnection will open a new connection.
+ */
+ if (PQstatus(entry->conn) != CONNECTION_OK ||
+ PQtransactionStatus(entry->conn) != PQTRANS_IDLE ||
+ entry->changing_xact_state ||
+ entry->invalidated ||
+ !entry->keep_connections)
+ {
+ elog(DEBUG3, "discarding connection %p", entry->conn);
+ disconnect_pg_server(entry);
+ }
+}
+
/*
* Cancel the currently-in-progress query (whose query text we do not have)
* and ignore the result. Returns true if we successfully cancel the query
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index fd141a0fa5..54b473de3d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9452,7 +9452,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
@@ -10768,3 +10768,76 @@ ERROR: invalid value for integer option "batch_size": 100$%$#$#
ALTER FOREIGN DATA WRAPPER postgres_fdw OPTIONS (nonexistent 'fdw');
ERROR: invalid option "nonexistent"
HINT: There are no valid options in this context.
+-- ===================================================================
+-- test parallel commit
+-- ===================================================================
+ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+CREATE TABLE ploc1 (f1 int, f2 text);
+CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
+ SERVER loopback OPTIONS (table_name 'ploc1');
+CREATE TABLE ploc2 (f1 int, f2 text);
+CREATE FOREIGN TABLE prem2 (f1 int, f2 text)
+ SERVER loopback OPTIONS (table_name 'ploc2');
+BEGIN;
+INSERT INTO prem1 VALUES (101, 'foo');
+INSERT INTO prem2 VALUES (201, 'bar');
+COMMIT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+-----
+ 101 | foo
+(1 row)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+-----
+ 201 | bar
+(1 row)
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (102, 'foofoo');
+INSERT INTO prem2 VALUES (202, 'barbar');
+RELEASE SAVEPOINT s;
+COMMIT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+(2 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+(2 rows)
+
+-- This tests parallelizing crearing prepared statements during pre-commit
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (103, 'foofoo');
+INSERT INTO prem2 VALUES (203, 'barbar');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (104, 'test1');
+INSERT INTO prem2 VALUES (204, 'test2');
+COMMIT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | test1
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | test2
+(3 rows)
+
+ALTER SERVER loopback OPTIONS (DROP parallel_commit);
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index 48c7417e6e..1be5830c53 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -120,6 +120,7 @@ postgres_fdw_validator(PG_FUNCTION_ARGS)
strcmp(def->defname, "updatable") == 0 ||
strcmp(def->defname, "truncatable") == 0 ||
strcmp(def->defname, "async_capable") == 0 ||
+ strcmp(def->defname, "parallel_commit") == 0 ||
strcmp(def->defname, "keep_connections") == 0)
{
/* these accept only boolean values */
@@ -248,6 +249,7 @@ InitPgFdwOptions(void)
/* async_capable is available on both server and table */
{"async_capable", ForeignServerRelationId, false},
{"async_capable", ForeignTableRelationId, false},
+ {"parallel_commit", ForeignServerRelationId, false},
{"keep_connections", ForeignServerRelationId, false},
{"password_required", UserMappingRelationId, false},
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 43c30d492d..6bb8937ced 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3426,3 +3426,46 @@ CREATE FOREIGN TABLE inv_bsz (c1 int )
-- No option is allowed to be specified at foreign data wrapper level
ALTER FOREIGN DATA WRAPPER postgres_fdw OPTIONS (nonexistent 'fdw');
+
+-- ===================================================================
+-- test parallel commit
+-- ===================================================================
+ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+
+CREATE TABLE ploc1 (f1 int, f2 text);
+CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
+ SERVER loopback OPTIONS (table_name 'ploc1');
+CREATE TABLE ploc2 (f1 int, f2 text);
+CREATE FOREIGN TABLE prem2 (f1 int, f2 text)
+ SERVER loopback OPTIONS (table_name 'ploc2');
+
+BEGIN;
+INSERT INTO prem1 VALUES (101, 'foo');
+INSERT INTO prem2 VALUES (201, 'bar');
+COMMIT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (102, 'foofoo');
+INSERT INTO prem2 VALUES (202, 'barbar');
+RELEASE SAVEPOINT s;
+COMMIT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+-- This tests parallelizing crearing prepared statements during pre-commit
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (103, 'foofoo');
+INSERT INTO prem2 VALUES (203, 'barbar');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (104, 'test1');
+INSERT INTO prem2 VALUES (204, 'test2');
+COMMIT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+ALTER SERVER loopback OPTIONS (DROP parallel_commit);
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2021-11-01 06:22 ` Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
1 sibling, 1 reply; 22+ messages in thread
From: Fujii Masao @ 2021-11-01 06:22 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2021/10/31 18:05, Etsuro Fujita wrote:
> Hi,
>
> As I said before [1], I’m working on $SUBJECT. Attached is a WIP
> patch for that.
Thanks for the patch!
> The patch is pretty simple: if a server option added
> by the patch “parallel_commit” is enabled,
Could you tell me why the parameter is necessary?
Can't we always enable the feature?
> * RELEASE
> parallel_commit=0: 0.385 ms
> parallel_commit=1: 0.221 ms
>
> * COMMIT
> parallel_commit=0: 1.660 ms
> parallel_commit=1: 0.861 ms
>
> With the option enabled, the average latencies for both commands are
> reduced significantly!
Sounds great!
> I think we could extend this to abort cleanup of remote
> (sub)transactions during post-abort. Anyway, I think this is useful,
> so I’ll add this to the upcoming commitfest.
Thanks!
+ /* Consume whatever data is available from the socket */
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, sql);
Without the patch, PQconsumeInput() is not called before pgfdw_get_result()
But could you tell me why you added PQconsumeInput() there?
When ignore_errors argument is true, the error reported by
PQconsumeInput() should be ignored?
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
@ 2021-11-07 09:06 ` Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2021-11-07 09:06 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Mon, Nov 1, 2021 at 3:22 PM Fujii Masao <[email protected]> wrote:
> On 2021/10/31 18:05, Etsuro Fujita wrote:
> > The patch is pretty simple: if a server option added
> > by the patch “parallel_commit” is enabled,
>
> Could you tell me why the parameter is necessary?
> Can't we always enable the feature?
I don’t think parallel commit would cause performance degradation,
even in the case when there is just a single remote (sub)transaction
to commit when called from pgfdw_xact_callback
(pgfdw_subxact_callback), so I think it might be OK to enable it by
default. But my concern about doing so is the remote side: during
those functions, if there are a lot of (sub)transactions on a single
remote server that need to be committed, parallel commit would
increase the remote server’s load at (sub)transaction end than serial
commit, which is the existing implementation, as the requests to
commit those (sub)transactions are sent to the remote server at the
same time; which some users might want to avoid.
> > I think we could extend this to abort cleanup of remote
> > (sub)transactions during post-abort. Anyway, I think this is useful,
> > so I’ll add this to the upcoming commitfest.
>
> Thanks!
I'll update the patch as such in the next version.
> + /* Consume whatever data is available from the socket */
> + if (!PQconsumeInput(conn))
> + pgfdw_report_error(ERROR, NULL, conn, false, sql);
>
> Without the patch, PQconsumeInput() is not called before pgfdw_get_result()
> But could you tell me why you added PQconsumeInput() there?
The reason is that there might be the result already before calling
pgfdw_get_result(), in which case PQconsumeInput() followed by
PQisBusy() would allow us to call PQgetResult() without doing
WaitLatchOrSocket(), which I think is rather expensive.
> When ignore_errors argument is true, the error reported by
> PQconsumeInput() should be ignored?
I’m not sure about that, because the error might be caused by e.g.,
OOM in the local server, in which case I don’t think it is safe to
ignore it and continue the (sub)transaction-end processing.
Thanks for reviewing!
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2021-11-08 04:13 ` Fujii Masao <[email protected]>
2021-11-11 02:27 ` RE: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit [email protected] <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 2 replies; 22+ messages in thread
From: Fujii Masao @ 2021-11-08 04:13 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On 2021/11/07 18:06, Etsuro Fujita wrote:
> On Mon, Nov 1, 2021 at 3:22 PM Fujii Masao <[email protected]> wrote:
>> On 2021/10/31 18:05, Etsuro Fujita wrote:
>>> The patch is pretty simple: if a server option added
>>> by the patch “parallel_commit” is enabled,
>>
>> Could you tell me why the parameter is necessary?
>> Can't we always enable the feature?
>
> I don’t think parallel commit would cause performance degradation,
> even in the case when there is just a single remote (sub)transaction
> to commit when called from pgfdw_xact_callback
> (pgfdw_subxact_callback), so I think it might be OK to enable it by
> default. But my concern about doing so is the remote side: during
> those functions, if there are a lot of (sub)transactions on a single
> remote server that need to be committed, parallel commit would
> increase the remote server’s load at (sub)transaction end than serial
> commit, which is the existing implementation, as the requests to
> commit those (sub)transactions are sent to the remote server at the
> same time; which some users might want to avoid.
Thanks for explaining this! But probably I failed to get your point.
Sorry... Whichever parallel or serial commit approach, the number of
transactions to commit on the remote server is the same, isn't it?
For example, please imagine the case where a client requests
ten transactions per second to the local server. Each transaction
accesses to the foreign table, so which means that ten transaction
commit operations per second are requested to the remote server.
Unless I'm missing something, this number doesn't change whether
the foreign transaction is committed in parallel way or not.
Thought?
>>> I think we could extend this to abort cleanup of remote
>>> (sub)transactions during post-abort. Anyway, I think this is useful,
>>> so I’ll add this to the upcoming commitfest.
>>
>> Thanks!
>
> I'll update the patch as such in the next version.
IMO it's better to implement and commit these features gradually
if possible. Which would simplify the patch and make it
easier-to-review. So I think that it's better to implement
this feature as a separate patch.
>> + /* Consume whatever data is available from the socket */
>> + if (!PQconsumeInput(conn))
>> + pgfdw_report_error(ERROR, NULL, conn, false, sql);
>>
>> Without the patch, PQconsumeInput() is not called before pgfdw_get_result()
>> But could you tell me why you added PQconsumeInput() there?
>
> The reason is that there might be the result already before calling
> pgfdw_get_result(), in which case PQconsumeInput() followed by
> PQisBusy() would allow us to call PQgetResult() without doing
> WaitLatchOrSocket(), which I think is rather expensive.
Understood. It's helpful to add the comment about why PQconsumeInput()
is called there.
Also could you tell me how much expensive it is?
>> When ignore_errors argument is true, the error reported by
>> PQconsumeInput() should be ignored?
>
> I’m not sure about that, because the error might be caused by e.g.,
> OOM in the local server, in which case I don’t think it is safe to
> ignore it and continue the (sub)transaction-end processing.
But the existing code ignores the error at all, doesn't it?
If it's unsafe to do that, probably we should fix that at first?
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 22+ messages in thread
* RE: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
@ 2021-11-11 02:27 ` [email protected] <[email protected]>
2021-11-17 04:00 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
1 sibling, 1 reply; 22+ messages in thread
From: [email protected] @ 2021-11-11 02:27 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; 'Fujii Masao' <[email protected]>
Dear Fujita-san,
I love your proposal because it will remove a bottleneck
for PostgreSQL build-in sharding.
I read your patch briefly and I think basically it's good.
Currently I have only one comment.
In your patch, postgres_fdw sends a COMMIT command to all entries in the hash table
and waits for the result without a timeout from the first entry.
I think this specification is good because it's very simple,
but if a COMMIT for a particular foreign server could take some time,
I thought it might be more efficient to stop waiting for results and look at the next entry.
This is how it works. First, we define a function similar to pgfdw_get_result()
so that we can specify the timeout time as an argument to WaitLatchOrSocket().
Then change the function called by do_sql_command_end () to the new one,
and change the callback function to skip if the result has not yet arrived
How is it? Is it an unnecessary assumption that COMMIT takes time? Or is this the next step?
I will put a PoC if needed.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-11 02:27 ` RE: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit [email protected] <[email protected]>
@ 2021-11-17 04:00 ` Etsuro Fujita <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Etsuro Fujita @ 2021-11-17 04:00 UTC (permalink / raw)
To: [email protected] <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Fujii Masao <[email protected]>
Kuroda-san,
On Thu, Nov 11, 2021 at 11:27 AM [email protected]
<[email protected]> wrote:
> I love your proposal because it will remove a bottleneck
> for PostgreSQL build-in sharding.
>
> I read your patch briefly and I think basically it's good.
Great! Thanks for reviewing!
> Currently I have only one comment.
>
> In your patch, postgres_fdw sends a COMMIT command to all entries in the hash table
> and waits for the result without a timeout from the first entry.
> I think this specification is good because it's very simple,
> but if a COMMIT for a particular foreign server could take some time,
> I thought it might be more efficient to stop waiting for results and look at the next entry.
> This is how it works. First, we define a function similar to pgfdw_get_result()
> so that we can specify the timeout time as an argument to WaitLatchOrSocket().
> Then change the function called by do_sql_command_end () to the new one,
> and change the callback function to skip if the result has not yet arrived
>
> How is it? Is it an unnecessary assumption that COMMIT takes time? Or is this the next step?
> I will put a PoC if needed.
Hmm, I'm not sure the cost-effectiveness of this optimization is
really high, because if the timeout expired, it means that something
unusual would have happened, and that it would take a long time for
the COMMIT command to complete (or abort at worst). So even if we
processed the rest of the entries while waiting for the command
result, we cannot reduce the total time very much. Maybe I'm missing
something, though.
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
@ 2021-11-16 09:55 ` Etsuro Fujita <[email protected]>
2021-11-18 04:09 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
1 sibling, 2 replies; 22+ messages in thread
From: Etsuro Fujita @ 2021-11-16 09:55 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Mon, Nov 8, 2021 at 1:13 PM Fujii Masao <[email protected]> wrote:
> On 2021/11/07 18:06, Etsuro Fujita wrote:
> > On Mon, Nov 1, 2021 at 3:22 PM Fujii Masao <[email protected]> wrote:
> >> Could you tell me why the parameter is necessary?
> >> Can't we always enable the feature?
> > I think it might be OK to enable it by
> > default. But my concern about doing so is the remote side: during
> > those functions, if there are a lot of (sub)transactions on a single
> > remote server that need to be committed, parallel commit would
> > increase the remote server’s load at (sub)transaction end than serial
> > commit, which is the existing implementation, as the requests to
> > commit those (sub)transactions are sent to the remote server at the
> > same time; which some users might want to avoid.
>
> Thanks for explaining this! But probably I failed to get your point.
> Sorry... Whichever parallel or serial commit approach, the number of
> transactions to commit on the remote server is the same, isn't it?
> For example, please imagine the case where a client requests
> ten transactions per second to the local server. Each transaction
> accesses to the foreign table, so which means that ten transaction
> commit operations per second are requested to the remote server.
> Unless I'm missing something, this number doesn't change whether
> the foreign transaction is committed in parallel way or not.
Sorry, my explanation was not enough, but I don’t think this is always
true. Let me explain using an example:
create server loopback foreign data wrapper postgres_fdw options
(dbname 'postgres', parallel_commit 'true');
create user mapping for current_user server loopback;
create table t1 (a int, b int);
create table t2 (a int, b int);
create foreign table ft1 (a int, b int) server loopback options
(table_name 't1');
create foreign table ft2 (a int, b int) server loopback options
(table_name 't2');
create role view_owner superuser;
create user mapping for view_owner server loopback;
grant SELECT on ft1 to view_owner;
create view v1 as select * from ft1;
alter view v1 owner to view_owner;
begin;
insert into v1 values (10, 10);
insert into ft2 values (20, 20);
commit;
For this transaction, since the first insert is executed as the view
owner while the second insert is executed as the current user, we
create a connection to the foreign server for each of the users to
execute the inserts. This leads to sending two commit commands to the
foreign server at the same time during pre-commit.
To avoid spike loads on a remote server induced by such a workload, I
think it’s a good idea to have a server option to control whether this
is enabled, but I might be too worried about that, so I want to hear
the opinions of people.
> IMO it's better to implement and commit these features gradually
> if possible. Which would simplify the patch and make it
> easier-to-review. So I think that it's better to implement
> this feature as a separate patch.
Ok, I'll create a patch for abort cleanup separately.
> >> + /* Consume whatever data is available from the socket */
> >> + if (!PQconsumeInput(conn))
> >> + pgfdw_report_error(ERROR, NULL, conn, false, sql);
> >>
> >> Without the patch, PQconsumeInput() is not called before pgfdw_get_result()
> >> But could you tell me why you added PQconsumeInput() there?
> >
> > The reason is that there might be the result already before calling
> > pgfdw_get_result(), in which case PQconsumeInput() followed by
> > PQisBusy() would allow us to call PQgetResult() without doing
> > WaitLatchOrSocket(), which I think is rather expensive.
>
> Understood. It's helpful to add the comment about why PQconsumeInput()
> is called there.
Ok.
> Also could you tell me how much expensive it is?
IIUC I think the overheads of WaitLatchOrSocket() incurred by a series
of epoll system calls are much larger compared to the overheads of
PQconsumeInput() incurred by a recv system call in non-blocking mode
when no data is available. I didn’t do testing, though.
Actually, we already use this optimization in libpqrcv_receive() for
the caller of that function to avoid doing WaitLatchOrSocket()?
> >> When ignore_errors argument is true, the error reported by
> >> PQconsumeInput() should be ignored?
> >
> > I’m not sure about that, because the error might be caused by e.g.,
> > OOM in the local server, in which case I don’t think it is safe to
> > ignore it and continue the (sub)transaction-end processing.
>
> But the existing code ignores the error at all, doesn't it?
> If it's unsafe to do that, probably we should fix that at first?
I changed my mind; I’ll update the patch to ignore the error as
before, because 1) as far as I know, there are no reports from the
field concerning that we ignore all kinds of errors in cleaning up the
prepared statements, so maybe we don’t need to change that, and 2) we
already committed at least one of the remote transactions, so it’s not
good to abort the local transaction unless we really have to.
Thanks!
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2021-11-18 04:09 ` Fujii Masao <[email protected]>
2021-11-18 12:45 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
1 sibling, 1 reply; 22+ messages in thread
From: Fujii Masao @ 2021-11-18 04:09 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On 2021/11/16 18:55, Etsuro Fujita wrote:
> Sorry, my explanation was not enough, but I don’t think this is always
> true. Let me explain using an example:
>
> create server loopback foreign data wrapper postgres_fdw options
> (dbname 'postgres', parallel_commit 'true');
> create user mapping for current_user server loopback;
> create table t1 (a int, b int);
> create table t2 (a int, b int);
> create foreign table ft1 (a int, b int) server loopback options
> (table_name 't1');
> create foreign table ft2 (a int, b int) server loopback options
> (table_name 't2');
> create role view_owner superuser;
> create user mapping for view_owner server loopback;
> grant SELECT on ft1 to view_owner;
> create view v1 as select * from ft1;
> alter view v1 owner to view_owner;
>
> begin;
> insert into v1 values (10, 10);
> insert into ft2 values (20, 20);
> commit;
>
> For this transaction, since the first insert is executed as the view
> owner while the second insert is executed as the current user, we
> create a connection to the foreign server for each of the users to
> execute the inserts. This leads to sending two commit commands to the
> foreign server at the same time during pre-commit.
>
> To avoid spike loads on a remote server induced by such a workload, I
> think it’s a good idea to have a server option to control whether this
> is enabled,
I understand your point. But even if the option is disabled (i.e.,
commit command is sent to each foreign server in serial way),
multiple queries still can run on the server concurrently and
which may cause performance "spike". Other clients may open several
sessions to the server and issue queries at the same time. Other
sessions using postgres_fdw may send commit command at the same time.
If we want to avoid that "spike", probably we need to decrease
max_connections or use connection pooling, etc. So ISTM that it's
half-baked and not enough to provide the option that controls
whether postgres_fdw issues commit command in parallel or serial way.
> but I might be too worried about that, so I want to hear
> the opinions of people.
Yes.
> IIUC I think the overheads of WaitLatchOrSocket() incurred by a series
> of epoll system calls are much larger compared to the overheads of
> PQconsumeInput() incurred by a recv system call in non-blocking mode
> when no data is available. I didn’t do testing, though.
Understood.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-18 04:09 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
@ 2021-11-18 12:45 ` Etsuro Fujita <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Etsuro Fujita @ 2021-11-18 12:45 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Thu, Nov 18, 2021 at 1:09 PM Fujii Masao <[email protected]> wrote:
> On 2021/11/16 18:55, Etsuro Fujita wrote:
> > Sorry, my explanation was not enough, but I don’t think this is always
> > true. Let me explain using an example:
> >
> > create server loopback foreign data wrapper postgres_fdw options
> > (dbname 'postgres', parallel_commit 'true');
> > create user mapping for current_user server loopback;
> > create table t1 (a int, b int);
> > create table t2 (a int, b int);
> > create foreign table ft1 (a int, b int) server loopback options
> > (table_name 't1');
> > create foreign table ft2 (a int, b int) server loopback options
> > (table_name 't2');
> > create role view_owner superuser;
> > create user mapping for view_owner server loopback;
> > grant SELECT on ft1 to view_owner;
> > create view v1 as select * from ft1;
> > alter view v1 owner to view_owner;
> >
> > begin;
> > insert into v1 values (10, 10);
> > insert into ft2 values (20, 20);
> > commit;
> >
> > For this transaction, since the first insert is executed as the view
> > owner while the second insert is executed as the current user, we
> > create a connection to the foreign server for each of the users to
> > execute the inserts. This leads to sending two commit commands to the
> > foreign server at the same time during pre-commit.
> >
> > To avoid spike loads on a remote server induced by such a workload, I
> > think it’s a good idea to have a server option to control whether this
> > is enabled,
>
> I understand your point. But even if the option is disabled (i.e.,
> commit command is sent to each foreign server in serial way),
> multiple queries still can run on the server concurrently and
> which may cause performance "spike". Other clients may open several
> sessions to the server and issue queries at the same time. Other
> sessions using postgres_fdw may send commit command at the same time.
> If we want to avoid that "spike", probably we need to decrease
> max_connections or use connection pooling, etc.
I think that what you are discussing here would be a related but
different issue, because the patch doesn't increase the number of
connections to the remote server that are needed for processing a
single transaction than before.
My concern about the patch is that in parallel-commit mode,
transactions like the above example might increase the remote server's
load at transaction end than before, while using the same number of
connections to the remote server as before, because multiple COMMIT
commands are sent to the remote server at the same time, not
sequentially as before. The option could be used to avoid such a
spike load without changing any settings on the remote server if
necessary. Also, the option could be added at no extra cost, so there
seems to me to be no reason to remove it.
Anyway, I'd like to hear the opinions of others.
Thanks!
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2021-12-03 09:07 ` Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
1 sibling, 1 reply; 22+ messages in thread
From: Fujii Masao @ 2021-12-03 09:07 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On 2021/11/16 18:55, Etsuro Fujita wrote:
> I changed my mind; I’ll update the patch to ignore the error as
> before, because 1) as far as I know, there are no reports from the
> field concerning that we ignore all kinds of errors in cleaning up the
> prepared statements, so maybe we don’t need to change that, and 2) we
> already committed at least one of the remote transactions, so it’s not
> good to abort the local transaction unless we really have to.
Are you planning to update the patch? In addition to this change,
at least documentation about new parallel_commit parameter needs
to be included in the patch.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
@ 2022-03-24 04:34 ` Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-03-24 04:34 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Sat, Mar 5, 2022 at 7:32 PM Etsuro Fujita <[email protected]> wrote:
> Attached is an updated version.
In the 0002 patch I introduced a macro for building an abort command
in preparation for the parallel abort patch (0003), but I moved it to
0003. Attached is a new patch set. The new version of 0002 is just a
cleanup patch (see the commit message in 0002), and I think it's
committable, so I'm planning to commit it, if no objections.
Best regards,
Etsuro Fujita
Attachments:
[application/octet-stream] v7-0002-postgres_fdw-Minor-cleanup-for-pgfdw_abort_cleanup.patch (3.7K, ../../CAPmGK16ABpu+ff1Wiwj3MztbDi2qFNmFEa_andc3LmVbCvADCw@mail.gmail.com/2-v7-0002-postgres_fdw-Minor-cleanup-for-pgfdw_abort_cleanup.patch)
download | inline diff:
From 54a26d1e652f85a3fee23aa4d7b2849ceb0487f1 Mon Sep 17 00:00:00 2001
From: Etsuro Fujita <[email protected]>
Date: Thu, 24 Mar 2022 13:16:09 +0900
Subject: [PATCH] postgres_fdw: Minor cleanup for pgfdw_abort_cleanup().
Commit 85c696112 introduced this function to deduplicate code in the
transaction callback functions, but the SQL command passed as an
argument to it was useless when it returned before aborting a remote
transaction using the command. Modify pgfdw_abort_cleanup() so that it
constructs the command when/if necessary, as before, removing the
argument from it. Also update comments in both pgfdw_abort_cleanup()
and the calling function.
Etsuro Fujita, reviewed by David Zhang.
Discussion: https://postgr.es/m/CAPmGK158hrd%3DZfXmgkmNFHivgh18e4oE2Gz151C2Q4OBDjZ08A%40mail.gmail.com
---
contrib/postgres_fdw/connection.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 74d3e73205..129ca79221 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -110,8 +110,7 @@ static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
static bool pgfdw_get_cleanup_result(PGconn *conn, TimestampTz endtime,
PGresult **result, bool *timed_out);
-static void pgfdw_abort_cleanup(ConnCacheEntry *entry, const char *sql,
- bool toplevel);
+static void pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel);
static void pgfdw_finish_pre_commit_cleanup(List *pending_entries);
static void pgfdw_finish_pre_subcommit_cleanup(List *pending_entries,
int curlevel);
@@ -1015,8 +1014,8 @@ pgfdw_xact_callback(XactEvent event, void *arg)
break;
case XACT_EVENT_PARALLEL_ABORT:
case XACT_EVENT_ABORT:
-
- pgfdw_abort_cleanup(entry, "ABORT TRANSACTION", true);
+ /* Rollback all remote transactions during abort */
+ pgfdw_abort_cleanup(entry, true);
break;
}
}
@@ -1109,10 +1108,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
else
{
/* Rollback all remote subtransactions during abort */
- snprintf(sql, sizeof(sql),
- "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
- curlevel, curlevel);
- pgfdw_abort_cleanup(entry, sql, false);
+ pgfdw_abort_cleanup(entry, false);
}
/* OK, we're outta that level of subtransaction */
@@ -1465,10 +1461,7 @@ exit: ;
}
/*
- * Abort remote transaction.
- *
- * The statement specified in "sql" is sent to the remote server,
- * in order to rollback the remote transaction.
+ * Abort remote transaction or subtransaction.
*
* "toplevel" should be set to true if toplevel (main) transaction is
* rollbacked, false otherwise.
@@ -1476,8 +1469,10 @@ exit: ;
* Set entry->changing_xact_state to false on success, true on failure.
*/
static void
-pgfdw_abort_cleanup(ConnCacheEntry *entry, const char *sql, bool toplevel)
+pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
{
+ char sql[100];
+
/*
* Don't try to clean up the connection if we're already in error
* recursion trouble.
@@ -1509,8 +1504,14 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, const char *sql, bool toplevel)
!pgfdw_cancel_query(entry->conn))
return; /* Unable to cancel running query */
+ if (toplevel)
+ snprintf(sql, sizeof(sql), "ABORT TRANSACTION");
+ else
+ snprintf(sql, sizeof(sql),
+ "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
+ entry->xact_depth, entry->xact_depth);
if (!pgfdw_exec_cleanup_query(entry->conn, sql, false))
- return; /* Unable to abort remote transaction */
+ return; /* Unable to abort remote (sub)transaction */
if (toplevel)
{
--
2.14.3 (Apple Git-98)
[application/octet-stream] v7-0003-postgres-fdw-Add-support-for-parallel-abort.patch (26.8K, ../../CAPmGK16ABpu+ff1Wiwj3MztbDi2qFNmFEa_andc3LmVbCvADCw@mail.gmail.com/3-v7-0003-postgres-fdw-Add-support-for-parallel-abort.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..a7672012f1 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -59,6 +59,7 @@ typedef struct ConnCacheEntry
bool have_error; /* have any subxacts aborted in this xact? */
bool changing_xact_state; /* xact state change in process */
bool parallel_commit; /* do we commit (sub)xacts in parallel? */
+ bool parallel_abort; /* do we abort (sub)xacts in parallel? */
bool invalidated; /* true if reconnect is pending */
bool keep_connections; /* setting value of keep_connections
* server option */
@@ -80,6 +81,21 @@ static unsigned int prep_stmt_number = 0;
/* tracks whether any work is needed in callback functions */
static bool xact_got_connection = false;
+/* Milliseconds to wait to cancel a query or execute a cleanup query */
+#define CONNECTION_CLEANUP_TIMEOUT 30000
+
+/* Macro for constructing abort command to be sent */
+#define CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel) \
+ do { \
+ if (toplevel) \
+ snprintf((sql), sizeof(sql), \
+ "ABORT TRANSACTION"); \
+ else \
+ snprintf((sql), sizeof(sql), \
+ "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d", \
+ (entry)->xact_depth, (entry)->xact_depth); \
+ } while(0)
+
/*
* SQL functions
*/
@@ -106,14 +122,28 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
+ bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
+static bool pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query);
+static bool pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime,
+ bool consume_input,
+ bool ignore_errors);
static bool pgfdw_get_cleanup_result(PGconn *conn, TimestampTz endtime,
PGresult **result, bool *timed_out);
static void pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel);
+static bool pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries,
+ List **cancel_requested);
static void pgfdw_finish_pre_commit_cleanup(List *pending_entries);
static void pgfdw_finish_pre_subcommit_cleanup(List *pending_entries,
int curlevel);
+static void pgfdw_finish_abort_cleanup(List *pending_entries,
+ List *cancel_requested,
+ bool toplevel);
static bool UserMappingPasswordRequired(UserMapping *user);
static bool disconnect_cached_connections(Oid serverid);
@@ -324,11 +354,12 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*
* By default, all the connections to any foreign servers are kept open.
*
- * Also determine whether to commit (sub)transactions opened on the remote
- * server in parallel at (sub)transaction end.
+ * Also determine whether to commit/abort (sub)transactions opened on the
+ * remote server in parallel at (sub)transaction end.
*/
entry->keep_connections = true;
entry->parallel_commit = false;
+ entry->parallel_abort = false;
foreach(lc, server->options)
{
DefElem *def = (DefElem *) lfirst(lc);
@@ -337,6 +368,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
entry->keep_connections = defGetBoolean(def);
else if (strcmp(def->defname, "parallel_commit") == 0)
entry->parallel_commit = defGetBoolean(def);
+ else if (strcmp(def->defname, "parallel_abort") == 0)
+ entry->parallel_abort = defGetBoolean(def);
}
/* Now try to make the connection */
@@ -922,6 +955,7 @@ pgfdw_xact_callback(XactEvent event, void *arg)
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Quick exit if no connections were touched in this transaction. */
if (!xact_got_connection)
@@ -1015,7 +1049,15 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_PARALLEL_ABORT:
case XACT_EVENT_ABORT:
/* Rollback all remote transactions during abort */
- pgfdw_abort_cleanup(entry, true);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, true,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, true);
break;
}
}
@@ -1025,11 +1067,21 @@ pgfdw_xact_callback(XactEvent event, void *arg)
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
- event == XACT_EVENT_PRE_COMMIT);
- pgfdw_finish_pre_commit_cleanup(pending_entries);
+ if (event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
+ event == XACT_EVENT_PRE_COMMIT)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_commit_cleanup(pending_entries);
+ }
+ else
+ {
+ Assert(event == XACT_EVENT_PARALLEL_ABORT ||
+ event == XACT_EVENT_ABORT);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ true);
+ }
}
/*
@@ -1054,6 +1106,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
ConnCacheEntry *entry;
int curlevel;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Nothing to do at subxact start, nor after commit. */
if (!(event == SUBXACT_EVENT_PRE_COMMIT_SUB ||
@@ -1108,7 +1161,15 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
else
{
/* Rollback all remote subtransactions during abort */
- pgfdw_abort_cleanup(entry, false);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, false,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, false);
}
/* OK, we're outta that level of subtransaction */
@@ -1116,10 +1177,19 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == SUBXACT_EVENT_PRE_COMMIT_SUB);
- pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ if (event == SUBXACT_EVENT_PRE_COMMIT_SUB)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ }
+ else
+ {
+ Assert(event == SUBXACT_EVENT_ABORT_SUB);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ false);
+ }
}
}
@@ -1263,17 +1333,25 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(conn))
+ return false;
+ return pgfdw_cancel_query_end(conn, endtime, false);
+}
+
+static bool
+pgfdw_cancel_query_begin(PGconn *conn)
+{
+ PGcancel *cancel;
+ char errbuf[256];
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
@@ -1293,6 +1371,31 @@ pgfdw_cancel_query(PGconn *conn)
PQfreeCancel(cancel);
}
+ return true;
+}
+
+static bool
+pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime, bool consume_input)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not get result of cancel request: %s",
+ pchomp(PQerrorMessage(conn)))));
+ return false;
+ }
+
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1327,9 +1430,7 @@ pgfdw_cancel_query(PGconn *conn)
static bool
pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
{
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to execute a cleanup query, assume the connection
@@ -1337,8 +1438,18 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
* place (e.g. statement timeout, user cancel), so the timeout shouldn't
* be too long.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_begin(conn, query))
+ return false;
+ return pgfdw_exec_cleanup_query_end(conn, query, endtime,
+ false, ignore_errors);
+}
+static bool
+pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query)
+{
/*
* Submit a query. Since we don't use non-blocking mode, this also can
* block. But its risk is relatively small, so we ignore that for now.
@@ -1349,6 +1460,30 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
return false;
}
+ return true;
+}
+
+static bool
+pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime, bool consume_input,
+ bool ignore_errors)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ pgfdw_report_error(WARNING, NULL, conn, false, query);
+ return false;
+ }
+
/* Get the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1504,12 +1639,7 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
!pgfdw_cancel_query(entry->conn))
return; /* Unable to cancel running query */
- if (toplevel)
- snprintf(sql, sizeof(sql), "ABORT TRANSACTION");
- else
- snprintf(sql, sizeof(sql),
- "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
- entry->xact_depth, entry->xact_depth);
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
if (!pgfdw_exec_cleanup_query(entry->conn, sql, false))
return; /* Unable to abort remote (sub)transaction */
@@ -1538,6 +1668,65 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
entry->changing_xact_state = false;
}
+/*
+ * Like pgfdw_abort_cleanup, submit an abort command or cancel request, but
+ * don't wait for the result.
+ *
+ * Returns true if the abort command or cancel request is successfully issued,
+ * false otherwise. If the abort command is successfully issued, the given
+ * connection cache entry is appended to *pending_entries. Othewise, if the
+ * cancel request is successfully issued, it's appended to *cancel_requested.
+ */
+static bool
+pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries, List **cancel_requested)
+{
+ /*
+ * Don't try to clean up the connection if we're already in error
+ * recursion trouble.
+ */
+ if (in_error_recursion_trouble())
+ entry->changing_xact_state = true;
+
+ /*
+ * If connection is already unsalvageable, don't touch it further.
+ */
+ if (entry->changing_xact_state)
+ return false;
+
+ /*
+ * Mark this connection as in the process of changing transaction state.
+ */
+ entry->changing_xact_state = true;
+
+ /* Assume we might have lost track of prepared statements */
+ entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by using an
+ * asynchronous execution function, the command might not have yet
+ * completed. Check to see if a command is still being processed by the
+ * remote server, and if so, request cancellation of the command.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ if (!pgfdw_cancel_query_begin(entry->conn))
+ return false; /* Unable to cancel running query */
+ *cancel_requested = lappend(*cancel_requested, entry);
+ }
+ else
+ {
+ char sql[100];
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ return false; /* Unable to abort remote transaction */
+ *pending_entries = lappend(*pending_entries, entry);
+ }
+
+ return true;
+}
+
/*
* Finish pre-commit cleanup of connections on each of which we've sent a
* COMMIT command to the remote server.
@@ -1644,6 +1833,168 @@ pgfdw_finish_pre_subcommit_cleanup(List *pending_entries, int curlevel)
}
}
+/*
+ * Finish abort cleanup of connections on each of which we've sent an abort
+ * command or cancel request to the remote server.
+ */
+static void
+pgfdw_finish_abort_cleanup(List *pending_entries, List *cancel_requested,
+ bool toplevel)
+{
+ List *pending_deallocs = NIL;
+ ListCell *lc;
+
+ /*
+ * For each of the pending cancel requests (if any), get and discard the
+ * result of the query, and submit an abort command to the remote server.
+ */
+ if (cancel_requested)
+ {
+ foreach(lc, cancel_requested)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. You might think we should do this before issuing
+ * cancel request like in normal mode, but that is problematic,
+ * because if, for example, it took longer than 30 seconds to
+ * process the first few entries in the cancel_requested list, it
+ * would cause a timeout error when processing each of the
+ * remaining entries in the list, leading to slamming that entry's
+ * connection shut.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_end(entry->conn, endtime, true))
+ {
+ /* Unable to cancel running query */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ /* Send an abort command in parallel if needed */
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_entries = lappend(pending_entries, entry);
+ }
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_entries)
+ return;
+
+ /*
+ * Get the result of the abort command for each of the pending entries
+ */
+ foreach(lc, pending_entries)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, sql, endtime,
+ true, false))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ if (toplevel)
+ {
+ /* Do a DEALLOCATE ALL in parallel if needed */
+ if (entry->have_prep_stmt && entry->have_error)
+ {
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn,
+ "DEALLOCATE ALL"))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_deallocs = lappend(pending_deallocs, entry);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+ }
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_deallocs)
+ return;
+ Assert(toplevel);
+
+ /*
+ * Get the result of the DEALLOCATE command for each of the pending
+ * entries
+ */
+ foreach(lc, pending_deallocs)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+
+ Assert(entry->changing_xact_state);
+ Assert(entry->have_prep_stmt);
+ Assert(entry->have_error);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, "DEALLOCATE ALL",
+ endtime, true, true))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+}
+
/*
* List active foreign server connections.
*
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index f210f91188..ed5bf9208f 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9509,7 +9509,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, parallel_abort, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
@@ -10934,10 +10934,12 @@ SELECT pg_terminate_backend(pid, 180000) FROM pg_stat_activity
RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
SERVER loopback OPTIONS (table_name 'ploc1');
@@ -11007,5 +11009,52 @@ SELECT * FROM prem2;
204 | quxqux
(3 rows)
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index 572591a558..59f865fac3 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -122,6 +122,7 @@ postgres_fdw_validator(PG_FUNCTION_ARGS)
strcmp(def->defname, "truncatable") == 0 ||
strcmp(def->defname, "async_capable") == 0 ||
strcmp(def->defname, "parallel_commit") == 0 ||
+ strcmp(def->defname, "parallel_abort") == 0 ||
strcmp(def->defname, "keep_connections") == 0)
{
/* these accept only boolean values */
@@ -251,6 +252,7 @@ InitPgFdwOptions(void)
{"async_capable", ForeignServerRelationId, false},
{"async_capable", ForeignTableRelationId, false},
{"parallel_commit", ForeignServerRelationId, false},
+ {"parallel_abort", ForeignServerRelationId, false},
{"keep_connections", ForeignServerRelationId, false},
{"password_required", UserMappingRelationId, false},
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 95b6b7192e..bd26739d9a 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3517,10 +3517,12 @@ RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
@@ -3559,5 +3561,26 @@ COMMIT;
SELECT * FROM prem1;
SELECT * FROM prem2;
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index d8dc715587..b86b3e9cc3 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -461,10 +461,10 @@ OPTIONS (ADD password_required 'false');
<para>
When multiple remote (sub)transactions are involved in a local
- (sub)transaction, by default <filename>postgres_fdw</filename> commits
- those remote (sub)transactions one by one when the local (sub)transaction
- commits.
- Performance can be improved with the following option:
+ (sub)transaction, by default <filename>postgres_fdw</filename> commits or
+ aborts those remote (sub)transactions one by one when the local
+ (sub)transaction commits or aborts.
+ Performance can be improved with the following options:
</para>
<variablelist>
@@ -479,27 +479,40 @@ OPTIONS (ADD password_required 'false');
This option can only be specified for foreign servers, not per-table.
The default is <literal>false</literal>.
</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>parallel_abort</literal> (<type>boolean</type>)</term>
+ <listitem>
<para>
- If multiple foreign servers with this option enabled are involved in
- a local (sub)transaction, multiple remote (sub)transactions opened on
- those foreign servers in the local (sub)transaction are committed in
- parallel across those foreign servers when the local (sub)transaction
- commits.
- </para>
-
- <para>
- For a foreign server with this option enabled, if many remote
- (sub)transactions are opened on the foreign server in a local
- (sub)transaction, this option might increase the remote server’s load
- when the local (sub)transaction commits, so be careful when using this
- option.
+ This option controls whether <filename>postgres_fdw</filename> aborts
+ remote (sub)transactions opened on a foreign server in a local
+ (sub)transaction in parallel when the local (sub)transaction aborts.
+ This option can only be specified for foreign servers, not per-table.
+ The default is <literal>false</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
+ <para>
+ If multiple foreign servers with these options enabled are involved in a
+ local (sub)transaction, multiple remote (sub)transactions opened on those
+ foreign servers in the local (sub)transaction are committed or aborted in
+ parallel across those foreign servers when the local (sub)transaction
+ commits or aborts.
+ </para>
+
+ <para>
+ For a foreign server with these options enabled, if many remote
+ (sub)transactions are opened on the foreign server in a local
+ (sub)transaction, these options might increase the remote server’s load
+ when the local (sub)transaction commits or aborts, so be careful when
+ using these options.
+ </para>
+
</sect3>
<sect3>
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-03-25 06:46 ` Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-03-25 06:46 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Thu, Mar 24, 2022 at 1:34 PM Etsuro Fujita <[email protected]> wrote:
> Attached is a new patch set. The new version of 0002 is just a
> cleanup patch (see the commit message in 0002), and I think it's
> committable, so I'm planning to commit it, if no objections.
Done.
Attached is the 0003 patch, which is the same as the one I sent yesterday.
Best regards,
Etsuro Fujita
Attachments:
[application/octet-stream] v7-0003-postgres-fdw-Add-support-for-parallel-abort.patch (26.8K, ../../CAPmGK16RWyc1nJsFnsn7-=UFzQiq4OGm70PK4-=5Y2fH_Gr4iA@mail.gmail.com/2-v7-0003-postgres-fdw-Add-support-for-parallel-abort.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 129ca79221..a7672012f1 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -59,6 +59,7 @@ typedef struct ConnCacheEntry
bool have_error; /* have any subxacts aborted in this xact? */
bool changing_xact_state; /* xact state change in process */
bool parallel_commit; /* do we commit (sub)xacts in parallel? */
+ bool parallel_abort; /* do we abort (sub)xacts in parallel? */
bool invalidated; /* true if reconnect is pending */
bool keep_connections; /* setting value of keep_connections
* server option */
@@ -80,6 +81,21 @@ static unsigned int prep_stmt_number = 0;
/* tracks whether any work is needed in callback functions */
static bool xact_got_connection = false;
+/* Milliseconds to wait to cancel a query or execute a cleanup query */
+#define CONNECTION_CLEANUP_TIMEOUT 30000
+
+/* Macro for constructing abort command to be sent */
+#define CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel) \
+ do { \
+ if (toplevel) \
+ snprintf((sql), sizeof(sql), \
+ "ABORT TRANSACTION"); \
+ else \
+ snprintf((sql), sizeof(sql), \
+ "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d", \
+ (entry)->xact_depth, (entry)->xact_depth); \
+ } while(0)
+
/*
* SQL functions
*/
@@ -106,14 +122,28 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
+ bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
+static bool pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query);
+static bool pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime,
+ bool consume_input,
+ bool ignore_errors);
static bool pgfdw_get_cleanup_result(PGconn *conn, TimestampTz endtime,
PGresult **result, bool *timed_out);
static void pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel);
+static bool pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries,
+ List **cancel_requested);
static void pgfdw_finish_pre_commit_cleanup(List *pending_entries);
static void pgfdw_finish_pre_subcommit_cleanup(List *pending_entries,
int curlevel);
+static void pgfdw_finish_abort_cleanup(List *pending_entries,
+ List *cancel_requested,
+ bool toplevel);
static bool UserMappingPasswordRequired(UserMapping *user);
static bool disconnect_cached_connections(Oid serverid);
@@ -324,11 +354,12 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*
* By default, all the connections to any foreign servers are kept open.
*
- * Also determine whether to commit (sub)transactions opened on the remote
- * server in parallel at (sub)transaction end.
+ * Also determine whether to commit/abort (sub)transactions opened on the
+ * remote server in parallel at (sub)transaction end.
*/
entry->keep_connections = true;
entry->parallel_commit = false;
+ entry->parallel_abort = false;
foreach(lc, server->options)
{
DefElem *def = (DefElem *) lfirst(lc);
@@ -337,6 +368,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
entry->keep_connections = defGetBoolean(def);
else if (strcmp(def->defname, "parallel_commit") == 0)
entry->parallel_commit = defGetBoolean(def);
+ else if (strcmp(def->defname, "parallel_abort") == 0)
+ entry->parallel_abort = defGetBoolean(def);
}
/* Now try to make the connection */
@@ -922,6 +955,7 @@ pgfdw_xact_callback(XactEvent event, void *arg)
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Quick exit if no connections were touched in this transaction. */
if (!xact_got_connection)
@@ -1015,7 +1049,15 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_PARALLEL_ABORT:
case XACT_EVENT_ABORT:
/* Rollback all remote transactions during abort */
- pgfdw_abort_cleanup(entry, true);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, true,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, true);
break;
}
}
@@ -1025,11 +1067,21 @@ pgfdw_xact_callback(XactEvent event, void *arg)
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
- event == XACT_EVENT_PRE_COMMIT);
- pgfdw_finish_pre_commit_cleanup(pending_entries);
+ if (event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
+ event == XACT_EVENT_PRE_COMMIT)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_commit_cleanup(pending_entries);
+ }
+ else
+ {
+ Assert(event == XACT_EVENT_PARALLEL_ABORT ||
+ event == XACT_EVENT_ABORT);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ true);
+ }
}
/*
@@ -1054,6 +1106,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
ConnCacheEntry *entry;
int curlevel;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Nothing to do at subxact start, nor after commit. */
if (!(event == SUBXACT_EVENT_PRE_COMMIT_SUB ||
@@ -1108,7 +1161,15 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
else
{
/* Rollback all remote subtransactions during abort */
- pgfdw_abort_cleanup(entry, false);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, false,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, false);
}
/* OK, we're outta that level of subtransaction */
@@ -1116,10 +1177,19 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == SUBXACT_EVENT_PRE_COMMIT_SUB);
- pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ if (event == SUBXACT_EVENT_PRE_COMMIT_SUB)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ }
+ else
+ {
+ Assert(event == SUBXACT_EVENT_ABORT_SUB);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ false);
+ }
}
}
@@ -1263,17 +1333,25 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(conn))
+ return false;
+ return pgfdw_cancel_query_end(conn, endtime, false);
+}
+
+static bool
+pgfdw_cancel_query_begin(PGconn *conn)
+{
+ PGcancel *cancel;
+ char errbuf[256];
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
@@ -1293,6 +1371,31 @@ pgfdw_cancel_query(PGconn *conn)
PQfreeCancel(cancel);
}
+ return true;
+}
+
+static bool
+pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime, bool consume_input)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not get result of cancel request: %s",
+ pchomp(PQerrorMessage(conn)))));
+ return false;
+ }
+
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1327,9 +1430,7 @@ pgfdw_cancel_query(PGconn *conn)
static bool
pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
{
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to execute a cleanup query, assume the connection
@@ -1337,8 +1438,18 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
* place (e.g. statement timeout, user cancel), so the timeout shouldn't
* be too long.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_begin(conn, query))
+ return false;
+ return pgfdw_exec_cleanup_query_end(conn, query, endtime,
+ false, ignore_errors);
+}
+static bool
+pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query)
+{
/*
* Submit a query. Since we don't use non-blocking mode, this also can
* block. But its risk is relatively small, so we ignore that for now.
@@ -1349,6 +1460,30 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
return false;
}
+ return true;
+}
+
+static bool
+pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime, bool consume_input,
+ bool ignore_errors)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ pgfdw_report_error(WARNING, NULL, conn, false, query);
+ return false;
+ }
+
/* Get the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1504,12 +1639,7 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
!pgfdw_cancel_query(entry->conn))
return; /* Unable to cancel running query */
- if (toplevel)
- snprintf(sql, sizeof(sql), "ABORT TRANSACTION");
- else
- snprintf(sql, sizeof(sql),
- "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
- entry->xact_depth, entry->xact_depth);
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
if (!pgfdw_exec_cleanup_query(entry->conn, sql, false))
return; /* Unable to abort remote (sub)transaction */
@@ -1538,6 +1668,65 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
entry->changing_xact_state = false;
}
+/*
+ * Like pgfdw_abort_cleanup, submit an abort command or cancel request, but
+ * don't wait for the result.
+ *
+ * Returns true if the abort command or cancel request is successfully issued,
+ * false otherwise. If the abort command is successfully issued, the given
+ * connection cache entry is appended to *pending_entries. Othewise, if the
+ * cancel request is successfully issued, it's appended to *cancel_requested.
+ */
+static bool
+pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries, List **cancel_requested)
+{
+ /*
+ * Don't try to clean up the connection if we're already in error
+ * recursion trouble.
+ */
+ if (in_error_recursion_trouble())
+ entry->changing_xact_state = true;
+
+ /*
+ * If connection is already unsalvageable, don't touch it further.
+ */
+ if (entry->changing_xact_state)
+ return false;
+
+ /*
+ * Mark this connection as in the process of changing transaction state.
+ */
+ entry->changing_xact_state = true;
+
+ /* Assume we might have lost track of prepared statements */
+ entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by using an
+ * asynchronous execution function, the command might not have yet
+ * completed. Check to see if a command is still being processed by the
+ * remote server, and if so, request cancellation of the command.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ if (!pgfdw_cancel_query_begin(entry->conn))
+ return false; /* Unable to cancel running query */
+ *cancel_requested = lappend(*cancel_requested, entry);
+ }
+ else
+ {
+ char sql[100];
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ return false; /* Unable to abort remote transaction */
+ *pending_entries = lappend(*pending_entries, entry);
+ }
+
+ return true;
+}
+
/*
* Finish pre-commit cleanup of connections on each of which we've sent a
* COMMIT command to the remote server.
@@ -1644,6 +1833,168 @@ pgfdw_finish_pre_subcommit_cleanup(List *pending_entries, int curlevel)
}
}
+/*
+ * Finish abort cleanup of connections on each of which we've sent an abort
+ * command or cancel request to the remote server.
+ */
+static void
+pgfdw_finish_abort_cleanup(List *pending_entries, List *cancel_requested,
+ bool toplevel)
+{
+ List *pending_deallocs = NIL;
+ ListCell *lc;
+
+ /*
+ * For each of the pending cancel requests (if any), get and discard the
+ * result of the query, and submit an abort command to the remote server.
+ */
+ if (cancel_requested)
+ {
+ foreach(lc, cancel_requested)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. You might think we should do this before issuing
+ * cancel request like in normal mode, but that is problematic,
+ * because if, for example, it took longer than 30 seconds to
+ * process the first few entries in the cancel_requested list, it
+ * would cause a timeout error when processing each of the
+ * remaining entries in the list, leading to slamming that entry's
+ * connection shut.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_end(entry->conn, endtime, true))
+ {
+ /* Unable to cancel running query */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ /* Send an abort command in parallel if needed */
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_entries = lappend(pending_entries, entry);
+ }
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_entries)
+ return;
+
+ /*
+ * Get the result of the abort command for each of the pending entries
+ */
+ foreach(lc, pending_entries)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, sql, endtime,
+ true, false))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ if (toplevel)
+ {
+ /* Do a DEALLOCATE ALL in parallel if needed */
+ if (entry->have_prep_stmt && entry->have_error)
+ {
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn,
+ "DEALLOCATE ALL"))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_deallocs = lappend(pending_deallocs, entry);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+ }
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_deallocs)
+ return;
+ Assert(toplevel);
+
+ /*
+ * Get the result of the DEALLOCATE command for each of the pending
+ * entries
+ */
+ foreach(lc, pending_deallocs)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+
+ Assert(entry->changing_xact_state);
+ Assert(entry->have_prep_stmt);
+ Assert(entry->have_error);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, "DEALLOCATE ALL",
+ endtime, true, true))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+}
+
/*
* List active foreign server connections.
*
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index f210f91188..ed5bf9208f 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9509,7 +9509,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, parallel_abort, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
@@ -10934,10 +10934,12 @@ SELECT pg_terminate_backend(pid, 180000) FROM pg_stat_activity
RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
SERVER loopback OPTIONS (table_name 'ploc1');
@@ -11007,5 +11009,52 @@ SELECT * FROM prem2;
204 | quxqux
(3 rows)
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index 572591a558..59f865fac3 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -122,6 +122,7 @@ postgres_fdw_validator(PG_FUNCTION_ARGS)
strcmp(def->defname, "truncatable") == 0 ||
strcmp(def->defname, "async_capable") == 0 ||
strcmp(def->defname, "parallel_commit") == 0 ||
+ strcmp(def->defname, "parallel_abort") == 0 ||
strcmp(def->defname, "keep_connections") == 0)
{
/* these accept only boolean values */
@@ -251,6 +252,7 @@ InitPgFdwOptions(void)
{"async_capable", ForeignServerRelationId, false},
{"async_capable", ForeignTableRelationId, false},
{"parallel_commit", ForeignServerRelationId, false},
+ {"parallel_abort", ForeignServerRelationId, false},
{"keep_connections", ForeignServerRelationId, false},
{"password_required", UserMappingRelationId, false},
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 95b6b7192e..bd26739d9a 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3517,10 +3517,12 @@ RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
@@ -3559,5 +3561,26 @@ COMMIT;
SELECT * FROM prem1;
SELECT * FROM prem2;
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index d8dc715587..b86b3e9cc3 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -461,10 +461,10 @@ OPTIONS (ADD password_required 'false');
<para>
When multiple remote (sub)transactions are involved in a local
- (sub)transaction, by default <filename>postgres_fdw</filename> commits
- those remote (sub)transactions one by one when the local (sub)transaction
- commits.
- Performance can be improved with the following option:
+ (sub)transaction, by default <filename>postgres_fdw</filename> commits or
+ aborts those remote (sub)transactions one by one when the local
+ (sub)transaction commits or aborts.
+ Performance can be improved with the following options:
</para>
<variablelist>
@@ -479,27 +479,40 @@ OPTIONS (ADD password_required 'false');
This option can only be specified for foreign servers, not per-table.
The default is <literal>false</literal>.
</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>parallel_abort</literal> (<type>boolean</type>)</term>
+ <listitem>
<para>
- If multiple foreign servers with this option enabled are involved in
- a local (sub)transaction, multiple remote (sub)transactions opened on
- those foreign servers in the local (sub)transaction are committed in
- parallel across those foreign servers when the local (sub)transaction
- commits.
- </para>
-
- <para>
- For a foreign server with this option enabled, if many remote
- (sub)transactions are opened on the foreign server in a local
- (sub)transaction, this option might increase the remote server’s load
- when the local (sub)transaction commits, so be careful when using this
- option.
+ This option controls whether <filename>postgres_fdw</filename> aborts
+ remote (sub)transactions opened on a foreign server in a local
+ (sub)transaction in parallel when the local (sub)transaction aborts.
+ This option can only be specified for foreign servers, not per-table.
+ The default is <literal>false</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
+ <para>
+ If multiple foreign servers with these options enabled are involved in a
+ local (sub)transaction, multiple remote (sub)transactions opened on those
+ foreign servers in the local (sub)transaction are committed or aborted in
+ parallel across those foreign servers when the local (sub)transaction
+ commits or aborts.
+ </para>
+
+ <para>
+ For a foreign server with these options enabled, if many remote
+ (sub)transactions are opened on the foreign server in a local
+ (sub)transaction, these options might increase the remote server’s load
+ when the local (sub)transaction commits or aborts, so be careful when
+ using these options.
+ </para>
+
</sect3>
<sect3>
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-05-04 21:38 ` David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: David Zhang @ 2022-05-04 21:38 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
Thanks a lot for the patch update.
On 2022-05-02 1:25 a.m., Etsuro Fujita wrote:
> Hi,
>
> On Wed, Apr 20, 2022 at 4:55 AM David Zhang <[email protected]> wrote:
>> I tried to apply the patch to master and plan to run some tests, but got
>> below errors due to other commits.
> I rebased the patch against HEAD. Attached is an updated patch.
Applied the patch v8 to master branch today, and the `make check` is OK.
I also repeated previous performance tests on three virtual Ubuntu
18.04, and the performance improvement of parallel abort in 10 times
average is more consistent.
before:
abort.1 = 2.6344 ms
abort.2 = 4.2799 ms
after:
abort.1 = 1.4105 ms
abort.2 = 2.2075 ms
>> + * remote server in parallel at (sub)transaction end.
>>
>> Here, I think the comment above could potentially apply to multiple
>> remote server(s).
> I agree on that point, but I think it's correct to say "the remote
> server" here, because we determine this for the given remote server.
> Maybe I'm missing something, so could you elaborate on it?
Your understanding is correct. I was thinking `remote server(s)` would
be easy for end user to understand but this is a comment in source code,
so either way is fine for me.
>> Not sure if there is a way to avoid repeated comments? For example, the
>> same comment below appears in two places (line 231 and line 296).
>>
>> + /*
>> + * If requested, consume whatever data is available from the socket.
>> + * (Note that if all data is available, this allows
>> + * pgfdw_get_cleanup_result to call PQgetResult without forcing the
>> + * overhead of WaitLatchOrSocket, which would be large compared to the
>> + * overhead of PQconsumeInput.)
>> + */
> IMO I think it's OK to have this in multiple places, because 1) IMO it
> wouldn't be that long, and 2) we already duplicated comments like this
> in the same file in v14 and earlier. Here is such an example in
> pgfdw_xact_callback() and pgfdw_subxact_callback() in that file in
> those versions:
>
> /*
> * If a command has been submitted to the remote server by
> * using an asynchronous execution function, the command
> * might not have yet completed. Check to see if a
> * command is still being processed by the remote server,
> * and if so, request cancellation of the command.
> */
>
> Thanks for reviewing! Sorry for the delay.
>
> Best regards,
> Etsuro Fujita
--
Best regards,
David
Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
@ 2022-05-06 10:08 ` Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-05-06 10:08 UTC (permalink / raw)
To: David Zhang <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, May 5, 2022 at 6:39 AM David Zhang <[email protected]> wrote:
> On 2022-05-02 1:25 a.m., Etsuro Fujita wrote:
> > On Wed, Apr 20, 2022 at 4:55 AM David Zhang <[email protected]> wrote:
> >> I tried to apply the patch to master and plan to run some tests, but got
> >> below errors due to other commits.
> > I rebased the patch against HEAD. Attached is an updated patch.
>
> Applied the patch v8 to master branch today, and the `make check` is OK.
> I also repeated previous performance tests on three virtual Ubuntu
> 18.04, and the performance improvement of parallel abort in 10 times
> average is more consistent.
>
> before:
>
> abort.1 = 2.6344 ms
> abort.2 = 4.2799 ms
>
> after:
>
> abort.1 = 1.4105 ms
> abort.2 = 2.2075 ms
Good to know! Thanks for testing!
> >> + * remote server in parallel at (sub)transaction end.
> >>
> >> Here, I think the comment above could potentially apply to multiple
> >> remote server(s).
> > I agree on that point, but I think it's correct to say "the remote
> > server" here, because we determine this for the given remote server.
> > Maybe I'm missing something, so could you elaborate on it?
> Your understanding is correct. I was thinking `remote server(s)` would
> be easy for end user to understand but this is a comment in source code,
> so either way is fine for me.
Ok, but I noticed that the comment failed to mention that the
parallel_commit option is disabled by default. Also, I noticed a
comment above it:
* It's enough to determine this only when making new connection because
* all the connections to the foreign server whose keep_connections option
* is changed will be closed and re-made later.
This would apply to the parallel_commit option as well. How about
updating these like the attached? (I simplified the latter comment
and moved it to a more appropriate place.)
Best regards,
Etsuro Fujita
Attachments:
[application/octet-stream] update-comment-in-make_new_connection.patch (1.1K, ../../CAPmGK16Kg2Bf90sqzcZ4YM5cN_G-4h7wFUS01qQpqNB+2BG5_w@mail.gmail.com/2-update-comment-in-make_new_connection.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index f9b8c01f3b..541526ab80 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -318,14 +318,15 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
* open even after the transaction using it ends, so that the subsequent
* transactions can re-use it.
*
- * It's enough to determine this only when making new connection because
- * all the connections to the foreign server whose keep_connections option
- * is changed will be closed and re-made later.
- *
* By default, all the connections to any foreign servers are kept open.
*
* Also determine whether to commit (sub)transactions opened on the remote
- * server in parallel at (sub)transaction end.
+ * server in parallel at (sub)transaction end, which is disabled by
+ * default.
+ *
+ * Note: it's enough to determine these only when making a new connection
+ * because these settings for it are changed, it will be closed and
+ * re-made later.
*/
entry->keep_connections = true;
entry->parallel_commit = false;
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-05-11 10:39 ` Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-05-11 10:39 UTC (permalink / raw)
To: David Zhang <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, May 6, 2022 at 7:08 PM Etsuro Fujita <[email protected]> wrote:
> > > On Wed, Apr 20, 2022 at 4:55 AM David Zhang <[email protected]> wrote:
> > >> + * remote server in parallel at (sub)transaction end.
> I noticed that the comment failed to mention that the
> parallel_commit option is disabled by default. Also, I noticed a
> comment above it:
>
> * It's enough to determine this only when making new connection because
> * all the connections to the foreign server whose keep_connections option
> * is changed will be closed and re-made later.
>
> This would apply to the parallel_commit option as well. How about
> updating these like the attached? (I simplified the latter comment
> and moved it to a more appropriate place.)
I’m planning to commit this as a follow-up patch for commit 04e706d42.
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-05-12 08:46 ` Etsuro Fujita <[email protected]>
2022-06-30 18:50 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Jacob Champion <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-05-12 08:46 UTC (permalink / raw)
To: David Zhang <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, May 11, 2022 at 7:39 PM Etsuro Fujita <[email protected]> wrote:
> On Fri, May 6, 2022 at 7:08 PM Etsuro Fujita <[email protected]> wrote:
> > > > On Wed, Apr 20, 2022 at 4:55 AM David Zhang <[email protected]> wrote:
> > > >> + * remote server in parallel at (sub)transaction end.
>
> > I noticed that the comment failed to mention that the
> > parallel_commit option is disabled by default. Also, I noticed a
> > comment above it:
> >
> > * It's enough to determine this only when making new connection because
> > * all the connections to the foreign server whose keep_connections option
> > * is changed will be closed and re-made later.
> >
> > This would apply to the parallel_commit option as well. How about
> > updating these like the attached? (I simplified the latter comment
> > and moved it to a more appropriate place.)
>
> I’m planning to commit this as a follow-up patch for commit 04e706d42.
Done.
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-06-30 18:50 ` Jacob Champion <[email protected]>
2022-07-07 08:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Jacob Champion @ 2022-06-30 18:50 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; David Zhang <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
On 5/12/22 01:46, Etsuro Fujita wrote:
> On Wed, May 11, 2022 at 7:39 PM Etsuro Fujita <[email protected]> wrote:
>> I’m planning to commit this as a follow-up patch for commit 04e706d42.
>
> Done.
FYI, I think cfbot is confused about the patch under review here. (When
I first opened the thread I thought the patch had already been committed.)
For new reviewers: it looks like v8, upthread, is the proposal.
--Jacob
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-06-30 18:50 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Jacob Champion <[email protected]>
@ 2022-07-07 08:22 ` Etsuro Fujita <[email protected]>
2022-09-30 20:54 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Etsuro Fujita @ 2022-07-07 08:22 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; +Cc: David Zhang <[email protected]>; Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi Jacob,
On Fri, Jul 1, 2022 at 3:50 AM Jacob Champion <[email protected]> wrote:
> On 5/12/22 01:46, Etsuro Fujita wrote:
> > On Wed, May 11, 2022 at 7:39 PM Etsuro Fujita <[email protected]> wrote:
> >> I’m planning to commit this as a follow-up patch for commit 04e706d42.
> >
> > Done.
>
> FYI, I think cfbot is confused about the patch under review here. (When
> I first opened the thread I thought the patch had already been committed.)
I should have attached the patch in the previous email.
> For new reviewers: it looks like v8, upthread, is the proposal.
The patch needs rebase due to commits 4036bcbbb, 8c8d307f8 and
82699edbf, so I updated the patch. Attached is a new version, in
which I also tweaked comments a little bit.
Thanks for taking care of this!
Best regards,
Etsuro Fujita
Attachments:
[application/octet-stream] v9-postgres-fdw-Add-support-for-parallel-abort.patch (27.3K, ../../CAPmGK16j4yiWnFngGAJM_05ZwPSZWHW0xwb_5YrACd40=AyJyQ@mail.gmail.com/2-v9-postgres-fdw-Add-support-for-parallel-abort.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index cffb6f8310..c2312c4e50 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -59,6 +59,7 @@ typedef struct ConnCacheEntry
bool have_error; /* have any subxacts aborted in this xact? */
bool changing_xact_state; /* xact state change in process */
bool parallel_commit; /* do we commit (sub)xacts in parallel? */
+ bool parallel_abort; /* do we abort (sub)xacts in parallel? */
bool invalidated; /* true if reconnect is pending */
bool keep_connections; /* setting value of keep_connections
* server option */
@@ -80,6 +81,25 @@ static unsigned int prep_stmt_number = 0;
/* tracks whether any work is needed in callback functions */
static bool xact_got_connection = false;
+/*
+ * Milliseconds to wait to cancel an in-progress query or execute a cleanup
+ * query; if it takes longer than 30 seconds to do these, we assume the
+ * connection is dead.
+ */
+#define CONNECTION_CLEANUP_TIMEOUT 30000
+
+/* Macro for constructing abort command to be sent */
+#define CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel) \
+ do { \
+ if (toplevel) \
+ snprintf((sql), sizeof(sql), \
+ "ABORT TRANSACTION"); \
+ else \
+ snprintf((sql), sizeof(sql), \
+ "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d", \
+ (entry)->xact_depth, (entry)->xact_depth); \
+ } while(0)
+
/*
* SQL functions
*/
@@ -106,14 +126,28 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
+ bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
+static bool pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query);
+static bool pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime,
+ bool consume_input,
+ bool ignore_errors);
static bool pgfdw_get_cleanup_result(PGconn *conn, TimestampTz endtime,
PGresult **result, bool *timed_out);
static void pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel);
+static bool pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries,
+ List **cancel_requested);
static void pgfdw_finish_pre_commit_cleanup(List *pending_entries);
static void pgfdw_finish_pre_subcommit_cleanup(List *pending_entries,
int curlevel);
+static void pgfdw_finish_abort_cleanup(List *pending_entries,
+ List *cancel_requested,
+ bool toplevel);
static bool UserMappingPasswordRequired(UserMapping *user);
static bool disconnect_cached_connections(Oid serverid);
@@ -320,8 +354,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*
* By default, all the connections to any foreign servers are kept open.
*
- * Also determine whether to commit (sub)transactions opened on the remote
- * server in parallel at (sub)transaction end, which is disabled by
+ * Also determine whether to commit/abort (sub)transactions opened on the
+ * remote server in parallel at (sub)transaction end, which is disabled by
* default.
*
* Note: it's enough to determine these only when making a new connection
@@ -330,6 +364,7 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*/
entry->keep_connections = true;
entry->parallel_commit = false;
+ entry->parallel_abort = false;
foreach(lc, server->options)
{
DefElem *def = (DefElem *) lfirst(lc);
@@ -338,6 +373,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
entry->keep_connections = defGetBoolean(def);
else if (strcmp(def->defname, "parallel_commit") == 0)
entry->parallel_commit = defGetBoolean(def);
+ else if (strcmp(def->defname, "parallel_abort") == 0)
+ entry->parallel_abort = defGetBoolean(def);
}
/* Now try to make the connection */
@@ -923,6 +960,7 @@ pgfdw_xact_callback(XactEvent event, void *arg)
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Quick exit if no connections were touched in this transaction. */
if (!xact_got_connection)
@@ -1016,7 +1054,15 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_PARALLEL_ABORT:
case XACT_EVENT_ABORT:
/* Rollback all remote transactions during abort */
- pgfdw_abort_cleanup(entry, true);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, true,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, true);
break;
}
}
@@ -1026,11 +1072,21 @@ pgfdw_xact_callback(XactEvent event, void *arg)
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
- event == XACT_EVENT_PRE_COMMIT);
- pgfdw_finish_pre_commit_cleanup(pending_entries);
+ if (event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
+ event == XACT_EVENT_PRE_COMMIT)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_commit_cleanup(pending_entries);
+ }
+ else
+ {
+ Assert(event == XACT_EVENT_PARALLEL_ABORT ||
+ event == XACT_EVENT_ABORT);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ true);
+ }
}
/*
@@ -1055,6 +1111,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
ConnCacheEntry *entry;
int curlevel;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Nothing to do at subxact start, nor after commit. */
if (!(event == SUBXACT_EVENT_PRE_COMMIT_SUB ||
@@ -1109,7 +1166,15 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
else
{
/* Rollback all remote subtransactions during abort */
- pgfdw_abort_cleanup(entry, false);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, false,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, false);
}
/* OK, we're outta that level of subtransaction */
@@ -1117,10 +1182,19 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == SUBXACT_EVENT_PRE_COMMIT_SUB);
- pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ if (event == SUBXACT_EVENT_PRE_COMMIT_SUB)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ }
+ else
+ {
+ Assert(event == SUBXACT_EVENT_ABORT_SUB);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ false);
+ }
}
}
@@ -1264,17 +1338,25 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(conn))
+ return false;
+ return pgfdw_cancel_query_end(conn, endtime, false);
+}
+
+static bool
+pgfdw_cancel_query_begin(PGconn *conn)
+{
+ PGcancel *cancel;
+ char errbuf[256];
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
@@ -1294,6 +1376,31 @@ pgfdw_cancel_query(PGconn *conn)
PQfreeCancel(cancel);
}
+ return true;
+}
+
+static bool
+pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime, bool consume_input)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not get result of cancel request: %s",
+ pchomp(PQerrorMessage(conn)))));
+ return false;
+ }
+
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1328,9 +1435,7 @@ pgfdw_cancel_query(PGconn *conn)
static bool
pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
{
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to execute a cleanup query, assume the connection
@@ -1338,8 +1443,18 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
* place (e.g. statement timeout, user cancel), so the timeout shouldn't
* be too long.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_begin(conn, query))
+ return false;
+ return pgfdw_exec_cleanup_query_end(conn, query, endtime,
+ false, ignore_errors);
+}
+static bool
+pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query)
+{
/*
* Submit a query. Since we don't use non-blocking mode, this also can
* block. But its risk is relatively small, so we ignore that for now.
@@ -1350,6 +1465,30 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
return false;
}
+ return true;
+}
+
+static bool
+pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime, bool consume_input,
+ bool ignore_errors)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ pgfdw_report_error(WARNING, NULL, conn, false, query);
+ return false;
+ }
+
/* Get the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1505,12 +1644,7 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
!pgfdw_cancel_query(entry->conn))
return; /* Unable to cancel running query */
- if (toplevel)
- snprintf(sql, sizeof(sql), "ABORT TRANSACTION");
- else
- snprintf(sql, sizeof(sql),
- "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
- entry->xact_depth, entry->xact_depth);
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
if (!pgfdw_exec_cleanup_query(entry->conn, sql, false))
return; /* Unable to abort remote (sub)transaction */
@@ -1539,6 +1673,65 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
entry->changing_xact_state = false;
}
+/*
+ * Like pgfdw_abort_cleanup, submit an abort command or cancel a request, but
+ * don't wait for the result.
+ *
+ * Returns true if the abort command or cancel request is successfully issued,
+ * false otherwise. If the abort command is successfully issued, the given
+ * connection cache entry is appended to *pending_entries. Othewise, if the
+ * cancel request is successfully issued, it's appended to *cancel_requested.
+ */
+static bool
+pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries, List **cancel_requested)
+{
+ /*
+ * Don't try to clean up the connection if we're already in error
+ * recursion trouble.
+ */
+ if (in_error_recursion_trouble())
+ entry->changing_xact_state = true;
+
+ /*
+ * If connection is already unsalvageable, don't touch it further.
+ */
+ if (entry->changing_xact_state)
+ return false;
+
+ /*
+ * Mark this connection as in the process of changing transaction state.
+ */
+ entry->changing_xact_state = true;
+
+ /* Assume we might have lost track of prepared statements */
+ entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by using an
+ * asynchronous execution function, the command might not have yet
+ * completed. Check to see if a command is still being processed by the
+ * remote server, and if so, request cancellation of the command.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ if (!pgfdw_cancel_query_begin(entry->conn))
+ return false; /* Unable to cancel running query */
+ *cancel_requested = lappend(*cancel_requested, entry);
+ }
+ else
+ {
+ char sql[100];
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ return false; /* Unable to abort remote transaction */
+ *pending_entries = lappend(*pending_entries, entry);
+ }
+
+ return true;
+}
+
/*
* Finish pre-commit cleanup of connections on each of which we've sent a
* COMMIT command to the remote server.
@@ -1647,6 +1840,168 @@ pgfdw_finish_pre_subcommit_cleanup(List *pending_entries, int curlevel)
}
}
+/*
+ * Finish abort cleanup of connections on each of which we've sent an abort
+ * command or cancel request to the remote server.
+ */
+static void
+pgfdw_finish_abort_cleanup(List *pending_entries, List *cancel_requested,
+ bool toplevel)
+{
+ List *pending_deallocs = NIL;
+ ListCell *lc;
+
+ /*
+ * For each of the pending cancel requests (if any), get and discard the
+ * result of the query, and submit an abort command to the remote server.
+ */
+ if (cancel_requested)
+ {
+ foreach(lc, cancel_requested)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. You might think we should do this before issuing
+ * cancel request like in normal mode, but that is problematic,
+ * because if, for example, it took longer than 30 seconds to
+ * process the first few entries in the cancel_requested list, it
+ * would cause a timeout error when processing each of the
+ * remaining entries in the list, leading to slamming that entry's
+ * connection shut.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_end(entry->conn, endtime, true))
+ {
+ /* Unable to cancel running query */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ /* Send an abort command in parallel if needed */
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_entries = lappend(pending_entries, entry);
+ }
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_entries)
+ return;
+
+ /*
+ * Get the result of the abort command for each of the pending entries
+ */
+ foreach(lc, pending_entries)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, sql, endtime,
+ true, false))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ if (toplevel)
+ {
+ /* Do a DEALLOCATE ALL in parallel if needed */
+ if (entry->have_prep_stmt && entry->have_error)
+ {
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn,
+ "DEALLOCATE ALL"))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_deallocs = lappend(pending_deallocs, entry);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+ }
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_deallocs)
+ return;
+ Assert(toplevel);
+
+ /*
+ * Get the result of the DEALLOCATE command for each of the pending
+ * entries
+ */
+ foreach(lc, pending_deallocs)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+
+ Assert(entry->changing_xact_state);
+ Assert(entry->have_prep_stmt);
+ Assert(entry->have_error);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, "DEALLOCATE ALL",
+ endtime, true, true))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+}
+
/*
* List active foreign server connections.
*
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..c7ba7b7a27 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9529,7 +9529,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, parallel_abort, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
@@ -11218,10 +11218,12 @@ SELECT pg_terminate_backend(pid, 180000) FROM pg_stat_activity
RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
SERVER loopback OPTIONS (table_name 'ploc1');
@@ -11291,5 +11293,52 @@ SELECT * FROM prem2;
204 | quxqux
(3 rows)
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index 572591a558..59f865fac3 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -122,6 +122,7 @@ postgres_fdw_validator(PG_FUNCTION_ARGS)
strcmp(def->defname, "truncatable") == 0 ||
strcmp(def->defname, "async_capable") == 0 ||
strcmp(def->defname, "parallel_commit") == 0 ||
+ strcmp(def->defname, "parallel_abort") == 0 ||
strcmp(def->defname, "keep_connections") == 0)
{
/* these accept only boolean values */
@@ -251,6 +252,7 @@ InitPgFdwOptions(void)
{"async_capable", ForeignServerRelationId, false},
{"async_capable", ForeignTableRelationId, false},
{"parallel_commit", ForeignServerRelationId, false},
+ {"parallel_abort", ForeignServerRelationId, false},
{"keep_connections", ForeignServerRelationId, false},
{"password_required", UserMappingRelationId, false},
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 92d1212027..7c5573ba9a 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3591,10 +3591,12 @@ RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
@@ -3633,5 +3635,26 @@ COMMIT;
SELECT * FROM prem1;
SELECT * FROM prem2;
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index bfd344cdc0..7b8b959462 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -465,12 +465,13 @@ OPTIONS (ADD password_required 'false');
corresponding remote transactions, and subtransactions are managed by
creating corresponding remote subtransactions. When multiple remote
transactions are involved in the current local transaction, by default
- <filename>postgres_fdw</filename> commits those remote transactions
- serially when the local transaction is committed. When multiple remote
- subtransactions are involved in the current local subtransaction, by
- default <filename>postgres_fdw</filename> commits those remote
- subtransactions serially when the local subtransaction is committed.
- Performance can be improved with the following option:
+ <filename>postgres_fdw</filename> commits or aborts those remote
+ transactions serially when the local transaction is committed or aborted.
+ When multiple remote subtransactions are involved in the current local
+ subtransaction, by default <filename>postgres_fdw</filename> commits or
+ aborts those remote subtransactions serially when the local subtransaction
+ is committed or abortd.
+ Performance can be improved with the following options:
</para>
<variablelist>
@@ -486,24 +487,38 @@ OPTIONS (ADD password_required 'false');
specified for foreign servers, not per-table. The default is
<literal>false</literal>.
</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>parallel_abort</literal> (<type>boolean</type>)</term>
+ <listitem>
<para>
- If multiple foreign servers with this option enabled are involved in a
- local transaction, multiple remote transactions on those foreign
- servers are committed in parallel across those foreign servers when
- the local transaction is committed.
- </para>
-
- <para>
- When this option is enabled, a foreign server with many remote
- transactions may see a negative performance impact when the local
- transaction is committed.
+ This option controls whether <filename>postgres_fdw</filename> aborts
+ in parallel remote transactions opened on a foreign server in a local
+ transaction when the local transaction is aborted. This setting also
+ applies to remote and local subtransactions. This option can only be
+ specified for foreign servers, not per-table. The default is
+ <literal>false</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
+ <para>
+ If multiple foreign servers with these options enabled are involved in a
+ local transaction, multiple remote transactions on those foreign servers
+ are committed or aborted in parallel across those foreign servers when
+ the local transaction is committed or aborted.
+ </para>
+
+ <para>
+ When these options are enabled, a foreign server with many remote
+ transactions may see a negative performance impact when the local
+ transaction is committed or aborted.
+ </para>
+
</sect3>
<sect3>
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-06-30 18:50 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Jacob Champion <[email protected]>
2022-07-07 08:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2022-09-30 20:54 ` David Zhang <[email protected]>
2022-11-01 10:24 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: David Zhang @ 2022-09-30 20:54 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; Jacob Champion <[email protected]>; +Cc: Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi Etsuro,
> The patch needs rebase due to commits 4036bcbbb, 8c8d307f8 and
> 82699edbf, so I updated the patch. Attached is a new version, in
> which I also tweaked comments a little bit.
After rebase the file `postgres_fdw.out` and applied to master branch,
make and make check are all ok for postgres_fdw.
--
David
Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-07 09:06 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2021-11-16 09:55 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Fujii Masao <[email protected]>
2022-03-24 04:34 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
2022-05-06 10:08 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-06-30 18:50 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Jacob Champion <[email protected]>
2022-07-07 08:22 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2022-09-30 20:54 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
@ 2022-11-01 10:24 ` Etsuro Fujita <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Etsuro Fujita @ 2022-11-01 10:24 UTC (permalink / raw)
To: David Zhang <[email protected]>; +Cc: Jacob Champion <[email protected]>; Fujii Masao <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi David,
On Sat, Oct 1, 2022 at 5:54 AM David Zhang <[email protected]> wrote:
> After rebase the file `postgres_fdw.out` and applied to master branch,
> make and make check are all ok for postgres_fdw.
Thanks for testing! Attached is a rebased version of the patch.
Best regards,
Etsuro Fujita
Attachments:
[application/octet-stream] v10-postgres-fdw-Add-support-for-parallel-abort.patch (25.9K, ../../CAPmGK14BggQDrP-KuuNpC59bwbsUZ5Aohx+jFoU9ptgprBwStw@mail.gmail.com/2-v10-postgres-fdw-Add-support-for-parallel-abort.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index f0c45b00db..864f9516ad 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -59,6 +59,7 @@ typedef struct ConnCacheEntry
bool have_error; /* have any subxacts aborted in this xact? */
bool changing_xact_state; /* xact state change in process */
bool parallel_commit; /* do we commit (sub)xacts in parallel? */
+ bool parallel_abort; /* do we abort (sub)xacts in parallel? */
bool invalidated; /* true if reconnect is pending */
bool keep_connections; /* setting value of keep_connections
* server option */
@@ -80,6 +81,25 @@ static unsigned int prep_stmt_number = 0;
/* tracks whether any work is needed in callback functions */
static bool xact_got_connection = false;
+/*
+ * Milliseconds to wait to cancel an in-progress query or execute a cleanup
+ * query; if it takes longer than 30 seconds to do these, we assume the
+ * connection is dead.
+ */
+#define CONNECTION_CLEANUP_TIMEOUT 30000
+
+/* Macro for constructing abort command to be sent */
+#define CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel) \
+ do { \
+ if (toplevel) \
+ snprintf((sql), sizeof(sql), \
+ "ABORT TRANSACTION"); \
+ else \
+ snprintf((sql), sizeof(sql), \
+ "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d", \
+ (entry)->xact_depth, (entry)->xact_depth); \
+ } while(0)
+
/*
* SQL functions
*/
@@ -106,14 +126,28 @@ static void pgfdw_inval_callback(Datum arg, int cacheid, uint32 hashvalue);
static void pgfdw_reject_incomplete_xact_state_change(ConnCacheEntry *entry);
static void pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel);
static bool pgfdw_cancel_query(PGconn *conn);
+static bool pgfdw_cancel_query_begin(PGconn *conn);
+static bool pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime,
+ bool consume_input);
static bool pgfdw_exec_cleanup_query(PGconn *conn, const char *query,
bool ignore_errors);
+static bool pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query);
+static bool pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime,
+ bool consume_input,
+ bool ignore_errors);
static bool pgfdw_get_cleanup_result(PGconn *conn, TimestampTz endtime,
PGresult **result, bool *timed_out);
static void pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel);
+static bool pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries,
+ List **cancel_requested);
static void pgfdw_finish_pre_commit_cleanup(List *pending_entries);
static void pgfdw_finish_pre_subcommit_cleanup(List *pending_entries,
int curlevel);
+static void pgfdw_finish_abort_cleanup(List *pending_entries,
+ List *cancel_requested,
+ bool toplevel);
static bool UserMappingPasswordRequired(UserMapping *user);
static bool disconnect_cached_connections(Oid serverid);
@@ -320,8 +354,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*
* By default, all the connections to any foreign servers are kept open.
*
- * Also determine whether to commit (sub)transactions opened on the remote
- * server in parallel at (sub)transaction end, which is disabled by
+ * Also determine whether to commit/abort (sub)transactions opened on the
+ * remote server in parallel at (sub)transaction end, which is disabled by
* default.
*
* Note: it's enough to determine these only when making a new connection
@@ -330,6 +364,7 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
*/
entry->keep_connections = true;
entry->parallel_commit = false;
+ entry->parallel_abort = false;
foreach(lc, server->options)
{
DefElem *def = (DefElem *) lfirst(lc);
@@ -338,6 +373,8 @@ make_new_connection(ConnCacheEntry *entry, UserMapping *user)
entry->keep_connections = defGetBoolean(def);
else if (strcmp(def->defname, "parallel_commit") == 0)
entry->parallel_commit = defGetBoolean(def);
+ else if (strcmp(def->defname, "parallel_abort") == 0)
+ entry->parallel_abort = defGetBoolean(def);
}
/* Now try to make the connection */
@@ -923,6 +960,7 @@ pgfdw_xact_callback(XactEvent event, void *arg)
HASH_SEQ_STATUS scan;
ConnCacheEntry *entry;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Quick exit if no connections were touched in this transaction. */
if (!xact_got_connection)
@@ -1016,7 +1054,15 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_PARALLEL_ABORT:
case XACT_EVENT_ABORT:
/* Rollback all remote transactions during abort */
- pgfdw_abort_cleanup(entry, true);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, true,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, true);
break;
}
}
@@ -1026,11 +1072,21 @@ pgfdw_xact_callback(XactEvent event, void *arg)
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
- event == XACT_EVENT_PRE_COMMIT);
- pgfdw_finish_pre_commit_cleanup(pending_entries);
+ if (event == XACT_EVENT_PARALLEL_PRE_COMMIT ||
+ event == XACT_EVENT_PRE_COMMIT)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_commit_cleanup(pending_entries);
+ }
+ else
+ {
+ Assert(event == XACT_EVENT_PARALLEL_ABORT ||
+ event == XACT_EVENT_ABORT);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ true);
+ }
}
/*
@@ -1055,6 +1111,7 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
ConnCacheEntry *entry;
int curlevel;
List *pending_entries = NIL;
+ List *cancel_requested = NIL;
/* Nothing to do at subxact start, nor after commit. */
if (!(event == SUBXACT_EVENT_PRE_COMMIT_SUB ||
@@ -1109,7 +1166,15 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
else
{
/* Rollback all remote subtransactions during abort */
- pgfdw_abort_cleanup(entry, false);
+ if (entry->parallel_abort)
+ {
+ if (pgfdw_abort_cleanup_begin(entry, false,
+ &pending_entries,
+ &cancel_requested))
+ continue;
+ }
+ else
+ pgfdw_abort_cleanup(entry, false);
}
/* OK, we're outta that level of subtransaction */
@@ -1117,10 +1182,19 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
}
/* If there are any pending connections, finish cleaning them up */
- if (pending_entries)
+ if (pending_entries || cancel_requested)
{
- Assert(event == SUBXACT_EVENT_PRE_COMMIT_SUB);
- pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ if (event == SUBXACT_EVENT_PRE_COMMIT_SUB)
+ {
+ Assert(cancel_requested == NIL);
+ pgfdw_finish_pre_subcommit_cleanup(pending_entries, curlevel);
+ }
+ else
+ {
+ Assert(event == SUBXACT_EVENT_ABORT_SUB);
+ pgfdw_finish_abort_cleanup(pending_entries, cancel_requested,
+ false);
+ }
}
}
@@ -1264,17 +1338,25 @@ pgfdw_reset_xact_state(ConnCacheEntry *entry, bool toplevel)
static bool
pgfdw_cancel_query(PGconn *conn)
{
- PGcancel *cancel;
- char errbuf[256];
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to cancel the query and discard the result, assume
* the connection is dead.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_begin(conn))
+ return false;
+ return pgfdw_cancel_query_end(conn, endtime, false);
+}
+
+static bool
+pgfdw_cancel_query_begin(PGconn *conn)
+{
+ PGcancel *cancel;
+ char errbuf[256];
/*
* Issue cancel request. Unfortunately, there's no good way to limit the
@@ -1294,6 +1376,31 @@ pgfdw_cancel_query(PGconn *conn)
PQfreeCancel(cancel);
}
+ return true;
+}
+
+static bool
+pgfdw_cancel_query_end(PGconn *conn, TimestampTz endtime, bool consume_input)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not get result of cancel request: %s",
+ pchomp(PQerrorMessage(conn)))));
+ return false;
+ }
+
/* Get and discard the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1328,9 +1435,7 @@ pgfdw_cancel_query(PGconn *conn)
static bool
pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
{
- PGresult *result = NULL;
TimestampTz endtime;
- bool timed_out;
/*
* If it takes too long to execute a cleanup query, assume the connection
@@ -1338,8 +1443,18 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
* place (e.g. statement timeout, user cancel), so the timeout shouldn't
* be too long.
*/
- endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(), 30000);
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_begin(conn, query))
+ return false;
+ return pgfdw_exec_cleanup_query_end(conn, query, endtime,
+ false, ignore_errors);
+}
+static bool
+pgfdw_exec_cleanup_query_begin(PGconn *conn, const char *query)
+{
/*
* Submit a query. Since we don't use non-blocking mode, this also can
* block. But its risk is relatively small, so we ignore that for now.
@@ -1350,6 +1465,30 @@ pgfdw_exec_cleanup_query(PGconn *conn, const char *query, bool ignore_errors)
return false;
}
+ return true;
+}
+
+static bool
+pgfdw_exec_cleanup_query_end(PGconn *conn, const char *query,
+ TimestampTz endtime, bool consume_input,
+ bool ignore_errors)
+{
+ PGresult *result = NULL;
+ bool timed_out;
+
+ /*
+ * If requested, consume whatever data is available from the socket.
+ * (Note that if all data is available, this allows
+ * pgfdw_get_cleanup_result to call PQgetResult without forcing the
+ * overhead of WaitLatchOrSocket, which would be large compared to the
+ * overhead of PQconsumeInput.)
+ */
+ if (consume_input && !PQconsumeInput(conn))
+ {
+ pgfdw_report_error(WARNING, NULL, conn, false, query);
+ return false;
+ }
+
/* Get the result of the query. */
if (pgfdw_get_cleanup_result(conn, endtime, &result, &timed_out))
{
@@ -1505,12 +1644,7 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
!pgfdw_cancel_query(entry->conn))
return; /* Unable to cancel running query */
- if (toplevel)
- snprintf(sql, sizeof(sql), "ABORT TRANSACTION");
- else
- snprintf(sql, sizeof(sql),
- "ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
- entry->xact_depth, entry->xact_depth);
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
if (!pgfdw_exec_cleanup_query(entry->conn, sql, false))
return; /* Unable to abort remote (sub)transaction */
@@ -1539,6 +1673,65 @@ pgfdw_abort_cleanup(ConnCacheEntry *entry, bool toplevel)
entry->changing_xact_state = false;
}
+/*
+ * Like pgfdw_abort_cleanup, submit an abort command or cancel a request, but
+ * don't wait for the result.
+ *
+ * Returns true if the abort command or cancel request is successfully issued,
+ * false otherwise. If the abort command is successfully issued, the given
+ * connection cache entry is appended to *pending_entries. Othewise, if the
+ * cancel request is successfully issued, it's appended to *cancel_requested.
+ */
+static bool
+pgfdw_abort_cleanup_begin(ConnCacheEntry *entry, bool toplevel,
+ List **pending_entries, List **cancel_requested)
+{
+ /*
+ * Don't try to clean up the connection if we're already in error
+ * recursion trouble.
+ */
+ if (in_error_recursion_trouble())
+ entry->changing_xact_state = true;
+
+ /*
+ * If connection is already unsalvageable, don't touch it further.
+ */
+ if (entry->changing_xact_state)
+ return false;
+
+ /*
+ * Mark this connection as in the process of changing transaction state.
+ */
+ entry->changing_xact_state = true;
+
+ /* Assume we might have lost track of prepared statements */
+ entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by using an
+ * asynchronous execution function, the command might not have yet
+ * completed. Check to see if a command is still being processed by the
+ * remote server, and if so, request cancellation of the command.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ if (!pgfdw_cancel_query_begin(entry->conn))
+ return false; /* Unable to cancel running query */
+ *cancel_requested = lappend(*cancel_requested, entry);
+ }
+ else
+ {
+ char sql[100];
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ return false; /* Unable to abort remote transaction */
+ *pending_entries = lappend(*pending_entries, entry);
+ }
+
+ return true;
+}
+
/*
* Finish pre-commit cleanup of connections on each of which we've sent a
* COMMIT command to the remote server.
@@ -1647,6 +1840,168 @@ pgfdw_finish_pre_subcommit_cleanup(List *pending_entries, int curlevel)
}
}
+/*
+ * Finish abort cleanup of connections on each of which we've sent an abort
+ * command or cancel request to the remote server.
+ */
+static void
+pgfdw_finish_abort_cleanup(List *pending_entries, List *cancel_requested,
+ bool toplevel)
+{
+ List *pending_deallocs = NIL;
+ ListCell *lc;
+
+ /*
+ * For each of the pending cancel requests (if any), get and discard the
+ * result of the query, and submit an abort command to the remote server.
+ */
+ if (cancel_requested)
+ {
+ foreach(lc, cancel_requested)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. You might think we should do this before issuing
+ * cancel request like in normal mode, but that is problematic,
+ * because if, for example, it took longer than 30 seconds to
+ * process the first few entries in the cancel_requested list, it
+ * would cause a timeout error when processing each of the
+ * remaining entries in the list, leading to slamming that entry's
+ * connection shut.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_cancel_query_end(entry->conn, endtime, true))
+ {
+ /* Unable to cancel running query */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ /* Send an abort command in parallel if needed */
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn, sql))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_entries = lappend(pending_entries, entry);
+ }
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_entries)
+ return;
+
+ /*
+ * Get the result of the abort command for each of the pending entries
+ */
+ foreach(lc, pending_entries)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+ char sql[100];
+
+ Assert(entry->changing_xact_state);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ CONSTRUCT_ABORT_COMMAND(sql, entry, toplevel);
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, sql, endtime,
+ true, false))
+ {
+ /* Unable to abort remote (sub)transaction */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+
+ if (toplevel)
+ {
+ /* Do a DEALLOCATE ALL in parallel if needed */
+ if (entry->have_prep_stmt && entry->have_error)
+ {
+ if (!pgfdw_exec_cleanup_query_begin(entry->conn,
+ "DEALLOCATE ALL"))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+ else
+ pending_deallocs = lappend(pending_deallocs, entry);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+ }
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+
+ /* No further work if no pending entries */
+ if (!pending_deallocs)
+ return;
+ Assert(toplevel);
+
+ /*
+ * Get the result of the DEALLOCATE command for each of the pending
+ * entries
+ */
+ foreach(lc, pending_deallocs)
+ {
+ ConnCacheEntry *entry = (ConnCacheEntry *) lfirst(lc);
+ TimestampTz endtime;
+
+ Assert(entry->changing_xact_state);
+ Assert(entry->have_prep_stmt);
+ Assert(entry->have_error);
+
+ /*
+ * Set end time. We do this now, not before issuing the command like
+ * in normal mode, for the same reason as for the cancel_requested
+ * entries.
+ */
+ endtime = TimestampTzPlusMilliseconds(GetCurrentTimestamp(),
+ CONNECTION_CLEANUP_TIMEOUT);
+
+ if (!pgfdw_exec_cleanup_query_end(entry->conn, "DEALLOCATE ALL",
+ endtime, true, true))
+ {
+ /* Trouble clearing prepared statements */
+ pgfdw_reset_xact_state(entry, toplevel);
+ continue;
+ }
+ entry->have_prep_stmt = false;
+ entry->have_error = false;
+
+ /* Reset the per-connection state if needed */
+ if (entry->state.pendingAreq)
+ memset(&entry->state, 0, sizeof(entry->state));
+
+ /* We're done with this entry; unset the changing_xact_state flag */
+ entry->changing_xact_state = false;
+ pgfdw_reset_xact_state(entry, toplevel);
+ }
+}
+
/*
* List active foreign server connections.
*
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 558e94b845..5ddb483cd5 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11501,10 +11501,12 @@ SELECT pg_terminate_backend(pid, 180000) FROM pg_stat_activity
RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
SERVER loopback OPTIONS (table_name 'ploc1');
@@ -11574,5 +11576,52 @@ SELECT * FROM prem2;
204 | quxqux
(3 rows)
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+ f1 | f2
+-----+--------
+ 101 | foo
+ 102 | foofoo
+ 104 | bazbaz
+(3 rows)
+
+SELECT * FROM prem2;
+ f1 | f2
+-----+--------
+ 201 | bar
+ 202 | barbar
+ 204 | quxqux
+(3 rows)
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index fa80ee2a55..df6680fe27 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -125,6 +125,7 @@ postgres_fdw_validator(PG_FUNCTION_ARGS)
strcmp(def->defname, "truncatable") == 0 ||
strcmp(def->defname, "async_capable") == 0 ||
strcmp(def->defname, "parallel_commit") == 0 ||
+ strcmp(def->defname, "parallel_abort") == 0 ||
strcmp(def->defname, "keep_connections") == 0)
{
/* these accept only boolean values */
@@ -254,6 +255,7 @@ InitPgFdwOptions(void)
{"async_capable", ForeignServerRelationId, false},
{"async_capable", ForeignTableRelationId, false},
{"parallel_commit", ForeignServerRelationId, false},
+ {"parallel_abort", ForeignServerRelationId, false},
{"keep_connections", ForeignServerRelationId, false},
{"password_required", UserMappingRelationId, false},
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index b0dbb41fb5..4d5140669b 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3744,10 +3744,12 @@ RESET postgres_fdw.application_name;
RESET debug_discard_caches;
-- ===================================================================
--- test parallel commit
+-- test parallel commit and parallel abort
-- ===================================================================
ALTER SERVER loopback OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback OPTIONS (ADD parallel_abort 'true');
ALTER SERVER loopback2 OPTIONS (ADD parallel_commit 'true');
+ALTER SERVER loopback2 OPTIONS (ADD parallel_abort 'true');
CREATE TABLE ploc1 (f1 int, f2 text);
CREATE FOREIGN TABLE prem1 (f1 int, f2 text)
@@ -3786,5 +3788,26 @@ COMMIT;
SELECT * FROM prem1;
SELECT * FROM prem2;
+BEGIN;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
+BEGIN;
+SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ROLLBACK TO SAVEPOINT s;
+RELEASE SAVEPOINT s;
+INSERT INTO prem1 VALUES (105, 'test1');
+INSERT INTO prem2 VALUES (205, 'test2');
+ABORT;
+SELECT * FROM prem1;
+SELECT * FROM prem2;
+
ALTER SERVER loopback OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback OPTIONS (DROP parallel_abort);
ALTER SERVER loopback2 OPTIONS (DROP parallel_commit);
+ALTER SERVER loopback2 OPTIONS (DROP parallel_abort);
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index 527f4deaaa..2b309ad002 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -469,12 +469,13 @@ OPTIONS (ADD password_required 'false');
corresponding remote transactions, and subtransactions are managed by
creating corresponding remote subtransactions. When multiple remote
transactions are involved in the current local transaction, by default
- <filename>postgres_fdw</filename> commits those remote transactions
- serially when the local transaction is committed. When multiple remote
- subtransactions are involved in the current local subtransaction, by
- default <filename>postgres_fdw</filename> commits those remote
- subtransactions serially when the local subtransaction is committed.
- Performance can be improved with the following option:
+ <filename>postgres_fdw</filename> commits or aborts those remote
+ transactions serially when the local transaction is committed or aborted.
+ When multiple remote subtransactions are involved in the current local
+ subtransaction, by default <filename>postgres_fdw</filename> commits or
+ aborts those remote subtransactions serially when the local subtransaction
+ is committed or abortd.
+ Performance can be improved with the following options:
</para>
<variablelist>
@@ -490,24 +491,38 @@ OPTIONS (ADD password_required 'false');
specified for foreign servers, not per-table. The default is
<literal>false</literal>.
</para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>parallel_abort</literal> (<type>boolean</type>)</term>
+ <listitem>
<para>
- If multiple foreign servers with this option enabled are involved in a
- local transaction, multiple remote transactions on those foreign
- servers are committed in parallel across those foreign servers when
- the local transaction is committed.
- </para>
-
- <para>
- When this option is enabled, a foreign server with many remote
- transactions may see a negative performance impact when the local
- transaction is committed.
+ This option controls whether <filename>postgres_fdw</filename> aborts
+ in parallel remote transactions opened on a foreign server in a local
+ transaction when the local transaction is aborted. This setting also
+ applies to remote and local subtransactions. This option can only be
+ specified for foreign servers, not per-table. The default is
+ <literal>false</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
+ <para>
+ If multiple foreign servers with these options enabled are involved in a
+ local transaction, multiple remote transactions on those foreign servers
+ are committed or aborted in parallel across those foreign servers when
+ the local transaction is committed or aborted.
+ </para>
+
+ <para>
+ When these options are enabled, a foreign server with many remote
+ transactions may see a negative performance impact when the local
+ transaction is committed or aborted.
+ </para>
+
</sect3>
<sect3>
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
@ 2021-11-01 22:46 ` David Zhang <[email protected]>
2021-11-07 09:13 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
1 sibling, 1 reply; 22+ messages in thread
From: David Zhang @ 2021-11-01 22:46 UTC (permalink / raw)
To: Etsuro Fujita <[email protected]>; PostgreSQL Hackers <[email protected]>; +Cc: Fujii Masao <[email protected]>
> I evaluated the effectiveness of the patch using a simple
> multi-statement transaction:
>
> BEGIN;
> SAVEPOINT s;
> INSERT INTO ft1 VALUES (10, 10);
> INSERT INTO ft2 VALUES (20, 20);
> RELEASE SAVEPOINT s;
> COMMIT;
>
> where ft1 and ft2 are foreign tables created on different foreign
> servers hosted on different machines. I ran the transaction five
> times using the patch with the option enabled/disabled, and measured
> the latencies for the RELEASE and COMMIT commands in each run. The
> average latencies for these commands over the five runs are:
>
> * RELEASE
> parallel_commit=0: 0.385 ms
> parallel_commit=1: 0.221 ms
>
> * COMMIT
> parallel_commit=0: 1.660 ms
> parallel_commit=1: 0.861 ms
>
> With the option enabled, the average latencies for both commands are
> reduced significantly!
Followed your instructions, I performed some basic tests to compare the
performance between before and after. In my testing environment (two
foreign servers on the same local machine), the performance varies,
sometimes the time spent on RELEASE and COMMIT without patch are close
to after patched, but seems it always perform better after patched. Then
I ran a 1-millions tuples insert, 5 times average is something like below,
Before
RELEASE 0.171 ms, COMMIT 1.861 ms
After
RELEASE 0.147 ms, COMMIT 1.305 ms
Best regards,
--
David
Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 22:46 ` Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit David Zhang <[email protected]>
@ 2021-11-07 09:13 ` Etsuro Fujita <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Etsuro Fujita @ 2021-11-07 09:13 UTC (permalink / raw)
To: David Zhang <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Fujii Masao <[email protected]>
On Tue, Nov 2, 2021 at 7:47 AM David Zhang <[email protected]> wrote:
> Followed your instructions, I performed some basic tests to compare the
> performance between before and after. In my testing environment (two
> foreign servers on the same local machine), the performance varies,
> sometimes the time spent on RELEASE and COMMIT without patch are close
> to after patched, but seems it always perform better after patched. Then
> I ran a 1-millions tuples insert, 5 times average is something like below,
>
> Before
> RELEASE 0.171 ms, COMMIT 1.861 ms
>
> After
> RELEASE 0.147 ms, COMMIT 1.305 ms
Good to know! Thanks for testing!
Best regards,
Etsuro Fujita
^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2022-11-01 10:24 UTC | newest]
Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-10-31 09:05 postgres_fdw: commit remote (sub)transactions in parallel during pre-commit Etsuro Fujita <[email protected]>
2021-11-01 06:22 ` Fujii Masao <[email protected]>
2021-11-07 09:06 ` Etsuro Fujita <[email protected]>
2021-11-08 04:13 ` Fujii Masao <[email protected]>
2021-11-11 02:27 ` [email protected] <[email protected]>
2021-11-17 04:00 ` Etsuro Fujita <[email protected]>
2021-11-16 09:55 ` Etsuro Fujita <[email protected]>
2021-11-18 04:09 ` Fujii Masao <[email protected]>
2021-11-18 12:45 ` Etsuro Fujita <[email protected]>
2021-12-03 09:07 ` Fujii Masao <[email protected]>
2022-03-24 04:34 ` Etsuro Fujita <[email protected]>
2022-03-25 06:46 ` Etsuro Fujita <[email protected]>
2022-05-04 21:38 ` David Zhang <[email protected]>
2022-05-06 10:08 ` Etsuro Fujita <[email protected]>
2022-05-11 10:39 ` Etsuro Fujita <[email protected]>
2022-05-12 08:46 ` Etsuro Fujita <[email protected]>
2022-06-30 18:50 ` Jacob Champion <[email protected]>
2022-07-07 08:22 ` Etsuro Fujita <[email protected]>
2022-09-30 20:54 ` David Zhang <[email protected]>
2022-11-01 10:24 ` Etsuro Fujita <[email protected]>
2021-11-01 22:46 ` David Zhang <[email protected]>
2021-11-07 09:13 ` Etsuro Fujita <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox