pgpool2.git / summary / log / commit / refs
commit 693a6284cf2dfc2e60d5a2489c9ee5167defc3dc
Author: Tatsuo Ishii <ishii@sraoss.co.jp>
Date: Tue Dec 04 07:20:48 2018 +0000
Implement new feature not to accept incoming connections.
Currently Pgpool-II accepts up to num_init_children frontends and
queues up more connection requests until one of child process becomes
free. This mostly works well, but if each session take long time, the
queue grows long and the whole system does not work smoothly.
To overcome the problem, I propose a new way to deal with many
connection requests from frontend: just complain and refuses to
connect to Pgpool-II if number of num_init_children is already filled
up. This is exactly the same behavior as PostgreSQL.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2018-November/003153.html
Patch by Tatsuo Ishii.
doc.ja/src/sgml/connection-settings.sgml | 59 ++++++++++++++++++++++++++++--
doc/src/sgml/connection-settings.sgml | 38 ++++++++++++++++++-
src/config/pool_config_variables.c | 11 ++++++
src/include/pool_config.h | 1 +
src/protocol/child.c | 40 +++++++++++++++++---
src/sample/pgpool.conf.sample | 4 ++
src/sample/pgpool.conf.sample-logical | 4 ++
src/sample/pgpool.conf.sample-master-slave | 4 ++
src/sample/pgpool.conf.sample-replication | 4 ++
src/sample/pgpool.conf.sample-stream | 4 ++
src/utils/pool_process_reporting.c | 5 +++
11 files changed, 164 insertions(+), 10 deletions(-)
diff --git a/doc.ja/src/sgml/connection-settings.sgml b/doc.ja/src/sgml/connection-settings.sgml
index e66f2250c..7e051292f 100644
--- a/doc.ja/src/sgml/connection-settings.sgml
+++ b/doc.ja/src/sgml/connection-settings.sgml
@@ -223,14 +223,15 @@ PCPプロセスが接続を受け付けるUNIXドメインソケットを置く
<productname>Pgpool-II</productname>, <emphasis>they are
blocked (not rejected with an error, like <productname>PostgreSQL</>)
until a connection to any <productname>Pgpool-II</productname>
- process is closed</emphasis>. Up to
+ process is closed</emphasis> unless <xref linkend="guc-reserved-connections">
+ is set to 1 or more.. Up to
<xref linkend="guc-listen-backlog-multiplier">*
num_init_children can be queued.
-->
preforkする<productname>Pgpool-II</productname>のサーバプロセスの数です。
デフォルト値は32です。
num_init_childrenは<productname>Pgpool-II</productname>に対してクライアントが同時に接続できる上限の数でもあります。
-num_init_childrenより多いクライアントが<productname>Pgpool-II</productname>に接続しようとした場合、<emphasis>それらのクライアントは、<productname>Pgpool-II</productname>のどれかのプロセスへの接続が閉じられるまで待たされます(<productname>PostgreSQL</>のように接続拒否エラーにはなりません。)</emphasis>。
+num_init_childrenより多いクライアントが<productname>Pgpool-II</productname>に接続しようとした場合、<xref linkend="guc-reserved-connections">が1以上に設定されている場合を除き、<emphasis>それらのクライアントは、<productname>Pgpool-II</productname>のどれかのプロセスへの接続が閉じられるまで待たされます(<productname>PostgreSQL</>のように接続拒否エラーにはなりません。)</emphasis>。
待たされる数の上限は、<xref linkend="guc-listen-backlog-multiplier"> * num_init_children です。
</para>
@@ -327,8 +328,58 @@ max_pool*num_init_children*2 <= (max_connections - superuser_reserved_connect
このパラメータはサーバ起動時にのみ設定可能です。
</para>
</listitem>
- </varlistentry>
- </variablelist>
+ </varlistentry>
+
+ <varlistentry id="guc-reserved-connections" xreflabel="reserved_connections">
+ <term><varname>reserved_connections</varname> (<type>integer</type>)
+ <indexterm>
+<!--
+ <primary><varname>reserverd_connections</varname> configuration parameter</primary>
+-->
+ <primary><varname>reserverd_connections</varname>設定パラメータ</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+<!--
+ When this parameter is set to 1 or greater, incoming
+ connections from clients are not accepted with error
+ message "Sorry, too many clients already", rather than
+ blocked if the number of current connections from clients
+ is more than (<xref linkend="guc-num-init-children"> -
+ <varname>reserved_connections</varname>). For example,
+ if <varname>reserved_connections</varname> = 1
+ and <xref linkend="guc-num-init-children"> = 32, then the
+ 32th connection from a client will be refused. This
+ behavior is similar
+ to <productname>PostgreSQL</productname> and good for
+ systems on which the number of connections from clients is
+ large and each session may take long time. In this case
+ length of the listen queue could be very long and may
+ cause the system unstable. In this situation setting this
+ parameter to non 0 is a good idea to prevent the listen
+ queue becomes very long.
+-->
+このパラメータが1以上に設定されていると、(<xref linkend="guc-num-init-children"> - <varname>reserved_connections</varname>)以上のクライアントからの接続はブロックされるのではなく受け付けられず、"Sorry, too many clients already"というエラーになります。
+たとえば、<varname>reserved_connections</varname> = 1で、<xref linkend="guc-num-init-children"> = 32なら、32番目のクライアントからの接続は拒否されます。
+これは、<productname>PostgreSQL</productname>と似た挙動で、クライアントからの接続数が多く、各セッションが長時間に渡るようなシステムで有効です。
+この場合、listenキューが非常に長くなる可能性があり、システムを不安定にさせます。
+こうした状況では、このパラメータを0以外にして、listenキューが長くなるのを防ぐのは良い考えです。
+ </para>
+ <para>
+<!--
+ If this parameter is set to 0, no connection from clients
+ will be refused. The default value is 0.
+ This parameter can only be set at server start.
+-->
+このパラメータを0にすると、クライアントからの接続要求は拒否されなくなります。
+デフォルト値は0です。
+このパラメータはサーバ起動時にのみ設定可能です。
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
</sect2>
<sect2 id="runtime-config-authentication-settings">
diff --git a/doc/src/sgml/connection-settings.sgml b/doc/src/sgml/connection-settings.sgml
index 234f2d179..3178c308e 100644
--- a/doc/src/sgml/connection-settings.sgml
+++ b/doc/src/sgml/connection-settings.sgml
@@ -145,7 +145,9 @@
blocked (not rejected with an error,
like <productname>PostgreSQL</productname>) until a
connection to any <productname>Pgpool-II</productname>
- process is closed</emphasis>. Up to
+ process is closed</emphasis>
+ unless <xref linkend="guc-reserved-connections"> is set
+ to 1 or more. Up to
<xref linkend="guc-listen-backlog-multiplier">*
num_init_children can be queued.
</para>
@@ -207,6 +209,40 @@
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="guc-reserved-connections" xreflabel="reserved_connections">
+ <term><varname>reserved_connections</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>reserverd_connections</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ When this parameter is set to 1 or greater, incoming
+ connections from clients are not accepted with error
+ message "Sorry, too many clients already", rather than
+ blocked if the number of current connections from clients
+ is more than (<xref linkend="guc-num-init-children"> -
+ <varname>reserved_connections</varname>). For example,
+ if <varname>reserved_connections</varname> = 1
+ and <xref linkend="guc-num-init-children"> = 32, then the
+ 32th connection from a client will be refused. This
+ behavior is similar
+ to <productname>PostgreSQL</productname> and good for
+ systems on which the number of connections from clients is
+ large and each session may take long time. In this case
+ length of the listen queue could be very long and may
+ cause the system unstable. In this situation setting this
+ parameter to non 0 is a good idea to prevent the listen
+ queue becomes very long.
+ </para>
+ <para>
+ If this parameter is set to 0, no connection from clients
+ will be refused. The default value is 0.
+ This parameter can only be set at server start.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</sect2>
diff --git a/src/config/pool_config_variables.c b/src/config/pool_config_variables.c
index 99be52385..8a6c83b73 100644
--- a/src/config/pool_config_variables.c
+++ b/src/config/pool_config_variables.c
@@ -1519,6 +1519,17 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"reserved_connections", CFGCXT_INIT, CONNECTION_POOL_CONFIG,
+ "Number of reserved connections.",
+ CONFIG_VAR_TYPE_INT, false, 0
+ },
+ &g_pool_config.reserved_connections,
+ 0,
+ 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
{
{"listen_backlog_multiplier", CFGCXT_INIT, CONNECTION_CONFIG,
"length of connection queue from frontend to pgpool-II",
diff --git a/src/include/pool_config.h b/src/include/pool_config.h
index af3d153bb..f98e0cf2f 100644
--- a/src/include/pool_config.h
+++ b/src/include/pool_config.h
@@ -161,6 +161,7 @@ typedef struct
int num_init_children; /* # of children initially pre-forked */
int listen_backlog_multiplier; /* determines the size of the
* connection queue */
+ int reserved_connections; /* # of reserved connections */
bool serialize_accept; /* if non 0, serialize call to accept() to
* avoid thundering herd problem */
int child_life_time; /* if idle for this seconds, child exits */
diff --git a/src/protocol/child.c b/src/protocol/child.c
index 6869c5c6c..7dba3e9fa 100644
--- a/src/protocol/child.c
+++ b/src/protocol/child.c
@@ -68,7 +68,7 @@ static RETSIGTYPE reload_config_handler(int sig);
static RETSIGTYPE authentication_timeout(int sig);
static void send_params(POOL_CONNECTION * frontend, POOL_CONNECTION_POOL * backend);
static void send_frontend_exits(void);
-static void connection_count_up(void);
+static int connection_count_up(void);
static void connection_count_down(void);
static bool connect_using_existing_connection(POOL_CONNECTION * frontend,
POOL_CONNECTION_POOL * backend,
@@ -287,6 +287,7 @@ do_child(int *fds)
StartupPacket *sp;
int front_end_fd;
SockAddr saddr;
+ int con_count;
/* reset per iteration memory context */
MemoryContextSwitchTo(ProcessLoopContext);
@@ -317,7 +318,32 @@ do_child(int *fds)
if (front_end_fd == RETRY)
continue;
- connection_count_up();
+ /*
+ * Check if max connections from clients execeeded.
+ */
+ con_count = connection_count_up();
+ if (con_count > (pool_config->num_init_children - pool_config->reserved_connections))
+ {
+ POOL_CONNECTION * cp;
+ cp = pool_open(front_end_fd, false);
+ if (cp == NULL)
+ {
+ connection_count_down();
+ continue;
+ }
+ connection_count_down();
+ pool_send_fatal_message(cp, 3, "53300",
+ "Sorry, too many clients already",
+ "",
+ "",
+ __FILE__, __LINE__);
+ ereport(ERROR,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("Sorry, too many clients already")));
+ pool_close(cp);
+ continue;
+ }
+
accepted = 1;
check_config_reload();
@@ -1490,10 +1516,11 @@ discard_persistent_db_connection(POOL_CONNECTION_POOL_SLOT * cp)
}
/*
- * Count up connection counter (from frontend to pgpool)
- * in shared memory
+ * Count up connection counter (from frontend to pgpool) in shared memory and
+ * returns current counter value. Please note that the returned value may not
+ * be up to date since locking has been already released.
*/
-static void
+static int
connection_count_up(void)
{
pool_sigset_t oldmask;
@@ -1501,8 +1528,10 @@ connection_count_up(void)
POOL_SETMASK2(&BlockSig, &oldmask);
pool_semaphore_lock(CONN_COUNTER_SEM);
Req_info->conn_counter++;
+ elog(DEBUG5, "connection_count_up: number of connected children: %d", Req_info->conn_counter);
pool_semaphore_unlock(CONN_COUNTER_SEM);
POOL_SETMASK(&oldmask);
+ return Req_info->conn_counter;
}
/*
@@ -1526,6 +1555,7 @@ connection_count_down(void)
*/
if (Req_info->conn_counter > 0)
Req_info->conn_counter--;
+ elog(DEBUG5, "connection_count_down: number of connected children: %d", Req_info->conn_counter);
pool_semaphore_unlock(CONN_COUNTER_SEM);
POOL_SETMASK(&oldmask);
}
diff --git a/src/sample/pgpool.conf.sample b/src/sample/pgpool.conf.sample
index c5304e89d..2e7497a1c 100644
--- a/src/sample/pgpool.conf.sample
+++ b/src/sample/pgpool.conf.sample
@@ -43,6 +43,10 @@ listen_backlog_multiplier = 2
serialize_accept = off
# whether to serialize accept() call to avoid thundering herd problem
# (change requires restart)
+reserved_connections = 0
+ # Number of reserved connections.
+ # Pgpool-II does not accept connections if over
+ # num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
diff --git a/src/sample/pgpool.conf.sample-logical b/src/sample/pgpool.conf.sample-logical
index b3f1aeb68..24d7c539e 100644
--- a/src/sample/pgpool.conf.sample-logical
+++ b/src/sample/pgpool.conf.sample-logical
@@ -36,6 +36,10 @@ socket_dir = '/tmp'
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
+reserved_connections = 0
+ # Number of reserved connections.
+ # Pgpool-II does not accept connections if over
+ # num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
diff --git a/src/sample/pgpool.conf.sample-master-slave b/src/sample/pgpool.conf.sample-master-slave
index 9d330c270..f89adbfd7 100644
--- a/src/sample/pgpool.conf.sample-master-slave
+++ b/src/sample/pgpool.conf.sample-master-slave
@@ -43,6 +43,10 @@ listen_backlog_multiplier = 2
serialize_accept = off
# whether to serialize accept() call to avoid thundering herd problem
# (change requires restart)
+reserved_connections = 0
+ # Number of reserved connections.
+ # Pgpool-II does not accept connections if over
+ # num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
diff --git a/src/sample/pgpool.conf.sample-replication b/src/sample/pgpool.conf.sample-replication
index 827cea315..256db784b 100644
--- a/src/sample/pgpool.conf.sample-replication
+++ b/src/sample/pgpool.conf.sample-replication
@@ -43,6 +43,10 @@ listen_backlog_multiplier = 2
serialize_accept = off
# whether to serialize accept() call to avoid thundering herd problem
# (change requires restart)
+reserved_connections = 0
+ # Number of reserved connections.
+ # Pgpool-II does not accept connections if over
+ # num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
diff --git a/src/sample/pgpool.conf.sample-stream b/src/sample/pgpool.conf.sample-stream
index 84b791d36..4f55eff46 100644
--- a/src/sample/pgpool.conf.sample-stream
+++ b/src/sample/pgpool.conf.sample-stream
@@ -36,6 +36,10 @@ socket_dir = '/tmp'
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
+reserved_connections = 0
+ # Number of reserved connections.
+ # Pgpool-II does not accept connections if over
+ # num_init_chidlren - reserved_connections.
# - pgpool Communication Manager Connection Settings -
diff --git a/src/utils/pool_process_reporting.c b/src/utils/pool_process_reporting.c
index 328203b44..df93e66b1 100644
--- a/src/utils/pool_process_reporting.c
+++ b/src/utils/pool_process_reporting.c
@@ -275,6 +275,11 @@ get_config(int *nrows)
StrNCpy(status[i].desc, "whether to serialize accept() call", POOLCONFIG_MAXDESCLEN);
i++;
+ StrNCpy(status[i].name, "reserved_connections", POOLCONFIG_MAXNAMELEN);
+ snprintf(status[i].value, POOLCONFIG_MAXVALLEN, "%d", pool_config->reserved_connections);
+ StrNCpy(status[i].desc, "number of reserved connections", POOLCONFIG_MAXDESCLEN);
+ i++;
+
StrNCpy(status[i].name, "max_pool", POOLCONFIG_MAXNAMELEN);
snprintf(status[i].value, POOLCONFIG_MAXVALLEN, "%d", pool_config->max_pool);
StrNCpy(status[i].desc, "max # of connection pool per child", POOLCONFIG_MAXDESCLEN);
[parent: 169f7c3648be]