pgpool2.git / summary / log / commit / refs
commit c0eb0ee5d23e1a95b27084fdf881faa6bc594e6d
Author: Tatsuo Ishii <ishii@postgresql.org>
Date: Tue Sep 06 08:31:45 2016 +0000
Enhance performance of SELECT when lots of rows involved.
(From pgpool-hackers: 1784)
While taking care of bug track item 238:
http://www.pgpool.net/mantisbt/view.php?id=238
I found there's a room to enhance the performance of data transfer
from Pgpool-II to frontend. In an extreme case, such as sending many
rows to frontend, the enhancement was 47% to 62% according to my
benchmark result.
Currently Pgpool-II flushes data to network (calling write(2)) every
time it sends a row data ("Data Row" message) to frontend. For
example, if 10,000 rows needed to be transfer, 10,000 times write()s
are issued. This is pretty expensive. Since after repeating to send
row data, "Command Complete" message is sent, it's enough to issue a
write() with the command complete message. Also there are unnecessary
flushing are in handling the command complete message.
Attached patch tries to fix them. I did a quick benchmark on my laptop
(Ubuntu 14.04 LTS, 2 cores, 16GB mem, SSD) using pgbench (scale factor
= 1) + custom query (SELECT * FROM pgbench_accounts), with master
branch head and with patched version. The graph attached.
Unfortunately, performance in workloads where transferring few rows,
will not be enhanced since such rows are needed to flush to network
anyway.
src/protocol/CommandComplete.c | 8 ++------
src/protocol/pool_process_query.c | 17 ++++++++++++++++-
src/protocol/pool_proto_modules.c | 2 ++
3 files changed, 20 insertions(+), 7 deletions(-)
diff --git a/src/protocol/CommandComplete.c b/src/protocol/CommandComplete.c
index f18f3db11..832e5e4af 100644
--- a/src/protocol/CommandComplete.c
+++ b/src/protocol/CommandComplete.c
@@ -5,7 +5,7 @@
* pgpool: a language independent connection pool server for PostgreSQL
* written by Tatsuo Ishii
*
- * Copyright (c) 2003-2015 PgPool Global Development Group
+ * Copyright (c) 2003-2016 PgPool Global Development Group
*
* Permission to use, copy, modify, and distribute this software and
* its documentation for any purpose and without fee is hereby
@@ -370,17 +370,13 @@ static int foward_command_complete(POOL_CONNECTION *frontend, char *packet, int
{
int sendlen;
- pool_flush(frontend);
-
if (pool_write(frontend, "C", 1) < 0)
return -1;
- pool_flush(frontend);
-
sendlen = htonl(packetlen+4);
if (pool_write(frontend, &sendlen, sizeof(sendlen)) < 0)
return -1;
- pool_flush(frontend);
+
pool_write_and_flush(frontend, packet, packetlen);
return 0;
diff --git a/src/protocol/pool_process_query.c b/src/protocol/pool_process_query.c
index cc666b797..784165823 100644
--- a/src/protocol/pool_process_query.c
+++ b/src/protocol/pool_process_query.c
@@ -796,7 +796,22 @@ POOL_STATUS SimpleForwardToFrontend(char kind, POOL_CONNECTION *frontend,
pool_write(frontend, &kind, 1);
sendlen = htonl(len1+4);
pool_write(frontend, &sendlen, sizeof(sendlen));
- pool_write_and_flush(frontend, p1, len1);
+
+ /*
+ * Optimization for "Data Row" message. Since it is too often to receive
+ * and forward "Data Row" message, we do not flush the message to frontend
+ * now. We expect that "Command Complete" message (or "Error response" or
+ * "Notice response" message) follows the stream of data row message
+ * anyway, so flushing will be done at that time.
+ */
+ if (kind == 'D')
+ {
+ pool_write(frontend, p1, len1);
+ }
+ else
+ {
+ pool_write_and_flush(frontend, p1, len1);
+ }
ereport(DEBUG1,
(errmsg("SimpleForwardToFrontend: packet:%c length:%d",
diff --git a/src/protocol/pool_proto_modules.c b/src/protocol/pool_proto_modules.c
index c833f48a9..9549bbbb9 100644
--- a/src/protocol/pool_proto_modules.c
+++ b/src/protocol/pool_proto_modules.c
@@ -2637,8 +2637,10 @@ POOL_STATUS ProcessBackendResponse(POOL_CONNECTION *frontend,
default:
status = SimpleForwardToFrontend(kind, frontend, backend);
+#ifdef NOT_USED
if (pool_flush(frontend))
return POOL_END;
+#endif
break;
}
}
[parent: 455df3d586a6]