pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: etsuro.fujita@gmail.com
Cc: movead.li@highgo.ca
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: Asynchronous Append on postgres_fdw nodes.
Date: Tue, 9 Jun 2020 14:20:42 +0500
Message-ID: <90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru> (raw )
In-Reply-To: <20200604.150015.2305436578805492574.horikyota.ntt@gmail.com >
References: <158389119893.14910.7314550871261530973.pgcf@coridan.postgresql.org >
<CAPmGK161L4tgZ5vRrLtA+rmugKbtCq-9o6YhYA+s7oWG_FQ3dg@mail.gmail.com >
<c5b4ca1e-5550-a405-4199-191143f4e8c2@postgrespro.ru >
<20200604.150015.2305436578805492574.horikyota.ntt@gmail.com >
On 6/4/20 11:00 AM, Kyotaro Horiguchi wrote:
> Removed a useless variable PgFdwScanState.result_ready.
> Removed duplicate code from remove_async_node() by using move_to_next_waiter().
> Done some minor cleanups.
>
I am reviewing your code.
A couple of variables are no longer needed (see changes.patch in attachment.
Something about the cost of an asynchronous plan:
At the simple query plan (see below) I see:
1. Startup cost of local SeqScan is equal 0, ForeignScan - 100. But
startup cost of Append is 0.
2. Total cost of an Append node is a sum of the subplans. Maybe in the
case of asynchronous append we need to use some reduce factor?
explain select * from parts;
With Async Append:
=====================
Append (cost=0.00..2510.30 rows=106780 width=8)
Async subplans: 3
-> Async Foreign Scan on part_1 parts_2 (cost=100.00..177.80
rows=2260 width=8)
-> Async Foreign Scan on part_2 parts_3 (cost=100.00..177.80
rows=2260 width=8)
-> Async Foreign Scan on part_3 parts_4 (cost=100.00..177.80
rows=2260 width=8)
-> Seq Scan on part_0 parts_1 (cost=0.00..1443.00 rows=100000 width=8)
Without Async Append:
=====================
Append (cost=0.00..2510.30 rows=106780 width=8)
-> Seq Scan on part_0 parts_1 (cost=0.00..1443.00 rows=100000 width=8)
-> Foreign Scan on part_1 parts_2 (cost=100.00..177.80 rows=2260
width=8)
-> Foreign Scan on part_2 parts_3 (cost=100.00..177.80 rows=2260
width=8)
-> Foreign Scan on part_3 parts_4 (cost=100.00..177.80 rows=2260
width=8)
--
Andrey Lepikhov
Postgres Professional
https://postgrespro.com
The Russian Postgres Company
Attachments:
[text/x-patch] changes.patch (1.5K, ../90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru/2-changes.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index b04b6a0e54..4406a9c3b3 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1632,9 +1632,6 @@ remove_async_node(ForeignScanState *node)
if (leader == node)
{
- /* It's the leader */
- ForeignScanState *next_leader;
-
if (leader_state->s.commonstate->busy)
{
/*
@@ -1769,7 +1766,7 @@ postgresIterateForeignScan(ForeignScanState *node)
node->ss.ps.asyncstate = AS_AVAILABLE;
else
node->ss.ps.asyncstate = AS_WAITING;
-
+elog(WARNING, "No tuple result %d", fsstate->cursor_number);
return ExecClearTuple(slot);
}
@@ -3703,7 +3700,7 @@ request_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
-
+elog(WARNING, "FETCH: %s", sql);
if (!PQsendQuery(conn, sql))
pgfdw_report_error(ERROR, NULL, conn, false, sql);
@@ -3769,7 +3766,6 @@ fetch_received_data(ForeignScanState *node)
PG_TRY();
{
PGconn *conn = fsstate->s.conn;
- char sql[64];
int addrows;
size_t newsize;
int i;
@@ -3798,7 +3794,7 @@ fetch_received_data(ForeignScanState *node)
node,
fsstate->temp_cxt);
}
-
+elog(WARNING, "fetch cursor: %d (%d %d)", fsstate->cursor_number, fsstate->num_tuples, addrows);
/* Update fetch_ct_2 */
if (fsstate->fetch_ct_2 < 2 && fsstate->next_tuple == 0)
fsstate->fetch_ct_2++;
view thread (130+ messages) latest in thread
Message-ID: <90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru>
Permalink: ../90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru/
Also on: postgresql.org/message-id/90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru
copy link · copy postgr.es · copy git am
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.lepikhov@postgrespro.ru, horikyota.ntt@gmail.com, etsuro.fujita@gmail.com, movead.li@highgo.ca, pgsql-hackers@lists.postgresql.org
Subject: Re: Asynchronous Append on postgres_fdw nodes.
In-Reply-To: <90cced83-9061-0cac-0012-84e5faac8c78@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox