pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedPATCH: pgbench - option to build using ppoll() for larger connection counts
51+ messages / 10 participants
[nested] [flat]
* PATCH: pgbench - option to build using ppoll() for larger connection counts
@ 2017-09-25 18:01 Rady, Doug <radydoug@amazon.com>
2017-09-25 18:07 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2017-09-25 18:10 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-04 06:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-26 11:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
0 siblings, 4 replies; 51+ messages in thread
From: Rady, Doug @ 2017-09-25 18:01 UTC (permalink / raw)
To: pgsql-hackers
This patch enables building pgbench to use ppoll() instead of select()
to allow for more than (FD_SETSIZE - 10) connections. As implemented,
when using ppoll(), the only connection limitation is system resources.
The patch has been implemented to introduce a minimal of #ifdef/#ifndef
clutter in the code.
Two patches attached.
One based on REL9_6_STABLE.
One based on 'master' which can also apply to REL_10_STABLE.
doug
--
Doug Rady
Amazon Aurora PostgreSQL
radydoug@amazon.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
pgbench-96-ppoll.patch (application/octet-stream, 12.5K) download
diff --git a/configure b/configure
index 2821a8f7e4..c1ba3c1477 100755
--- a/configure
+++ b/configure
@@ -9053,6 +9053,62 @@ if test "$ac_res" != no; then :
fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing ppoll" >&5
+$as_echo_n "checking for library containing ppoll... " >&6; }
+if ${ac_cv_search_ppoll+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_func_search_save_LIBS=$LIBS
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char ppoll ();
+int
+main ()
+{
+return ppoll ();
+ ;
+ return 0;
+}
+_ACEOF
+for ac_lib in '' c; do
+ if test -z "$ac_lib"; then
+ ac_res="none required"
+ else
+ ac_res=-l$ac_lib
+ LIBS="-l$ac_lib $ac_func_search_save_LIBS"
+ fi
+ if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_search_ppoll=$ac_res
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext
+ if ${ac_cv_search_ppoll+:} false; then :
+ break
+fi
+done
+if ${ac_cv_search_ppoll+:} false; then :
+
+else
+ ac_cv_search_ppoll=no
+fi
+rm conftest.$ac_ext
+LIBS=$ac_func_search_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_ppoll" >&5
+$as_echo "$ac_cv_search_ppoll" >&6; }
+ac_res=$ac_cv_search_ppoll
+if test "$ac_res" != no; then :
+ test "$ac_res" = "none required" || LIBS="$ac_res $LIBS"
+
+fi
+
# Solaris:
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing fdatasync" >&5
$as_echo_n "checking for library containing fdatasync... " >&6; }
@@ -12517,7 +12573,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
+for ac_func in cbrt dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index a1696db921..b0b24867a6 100644
--- a/configure.in
+++ b/configure.in
@@ -1058,6 +1058,7 @@ AC_SEARCH_LIBS(getopt_long, [getopt gnugetopt])
AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(shm_open, rt)
AC_SEARCH_LIBS(shm_unlink, rt)
+AC_SEARCH_LIBS(ppoll, c)
# Solaris:
AC_SEARCH_LIBS(fdatasync, [rt posix4])
# Required for thread_test.c on Solaris
@@ -1457,7 +1458,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
+AC_CHECK_FUNCS([cbrt dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 163dcad137..9a6b95d05b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -46,6 +46,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -55,6 +58,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -89,6 +96,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_CLREV(s) { do { if ((s)->pfdp) { ((s)->pfdp)->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { memset((s)->pfdp, 0, sizeof(struct pollfd)); } } while(0); }
+#define PFD_SETFD(s) { do { ((s)->pfdp)->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_CLREV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -259,6 +287,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* transaction count */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -273,6 +302,7 @@ typedef struct
unsigned short random_state[3]; /* separate randomness for each thread */
int64 throttle_trigger; /* previous/next throttling (us) */
FILE *logfile; /* where to log, or NULL */
+ PFD_STRUCT_POLLFD(*pfds)
/* per thread collected stats */
instr_time start_time; /* thread start time */
@@ -386,6 +416,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -1734,11 +1765,7 @@ preparedStatementName(char *buffer, int file, int state)
static bool
clientDone(CState *st)
{
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return false; /* always false */
}
@@ -1914,10 +1941,7 @@ top:
if (commands[st->state + 1] == NULL)
{
if (is_connect)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
++st->cnt;
if ((st->cnt >= nxacts && duration <= 0) || timer_exceeded)
@@ -1968,6 +1992,7 @@ top:
st->sleeping = false;
st->throttling = false;
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_SETFD(st);
}
/*
@@ -2315,13 +2340,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/* create tables and setup data */
@@ -3495,7 +3514,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0 )
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -3756,6 +3779,14 @@ main(int argc, char **argv)
}
}
+ /*
+ * Don't need more threads than there are clients. (This is not merely an
+ * optimization; throttle_delay is calculated incorrectly below if some
+ * threads have no clients assigned to them.)
+ */
+ if (nthreads > nclients)
+ nthreads = nclients;
+
/* set default script if none */
if (num_scripts == 0 && !is_init_mode)
{
@@ -3779,14 +3810,6 @@ main(int argc, char **argv)
if (num_scripts > 1)
per_script_stats = true;
- /*
- * Don't need more threads than there are clients. (This is not merely an
- * optimization; throttle_delay is calculated incorrectly below if some
- * threads have no clients assigned to them.)
- */
- if (nthreads > nclients)
- nthreads = nclients;
-
/* compute a per thread delay */
throttle_delay *= nthreads;
@@ -4162,6 +4185,8 @@ threadRun(void *arg)
}
}
+ PFD_THREAD_INIT(thread, state, nstate);
+
if (!is_connect)
{
/* make connections to the database */
@@ -4169,6 +4194,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4192,28 +4218,32 @@ threadRun(void *arg)
fprintf(stderr, "client %d executing script \"%s\"\n", st->id,
sql_script[st->use_file].desc);
if (!doCustom(thread, st, &aggs))
+ {
remains--; /* I've aborted */
+ PFD_ZERO(st);
+ }
if (st->ecnt > prev_ecnt && commands[st->state]->type == META_COMMAND)
{
fprintf(stderr, "client %d aborted in state %d; execution of meta-command failed\n",
i, st->state);
remains--; /* I've aborted */
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
}
}
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited */
int64 now_usec = 0;
int64 min_usec;
+#ifndef USE_PPOLL
+ fd_set input_mask;
FD_ZERO(&input_mask);
-
maxsock = -1;
+#endif
+
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -4233,8 +4263,7 @@ threadRun(void *arg)
remains--;
st->sleeping = false;
st->throttling = false;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
continue;
}
else /* just a nap from the script */
@@ -4267,10 +4296,16 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ maxsock = 0;
+ (st->pfdp)->events = POLL_EVENTS;
+ (st->pfdp)->revents = 0;
+#else
FD_SET(sock, &input_mask);
if (maxsock < sock)
maxsock = sock;
+#endif
}
/* also wake up to print the next progress report on time */
@@ -4302,20 +4337,32 @@ threadRun(void *arg)
if (min_usec != PG_INT64_MAX)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
if (nsocks < 0)
{
if (errno == EINTR)
continue;
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
@@ -4337,18 +4384,36 @@ threadRun(void *arg)
PQerrorMessage(st->con));
goto done;
}
+#ifdef USE_PPOLL
+ if ((st->pfdp)->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x\n",
+ errno, i, ((st->pfdp)->revents & POLL_FAIL));
+ }
+
+ if (((st->pfdp)->revents) ||
+#else
if (FD_ISSET(sock, &input_mask) ||
+#endif
commands[st->state]->type == META_COMMAND)
{
if (!doCustom(thread, st, &aggs))
+ {
remains--; /* I've aborted */
+ PFD_ZERO(st);
+ }
}
+ PFD_CLREV(st);
}
else if (is_connect && st->sleeping)
{
/* it is sleeping for throttling, maybe it is done, let us try */
if (!doCustom(thread, st, &aggs))
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
if (st->ecnt > prev_ecnt && commands[st->state]->type == META_COMMAND)
@@ -4356,8 +4421,7 @@ threadRun(void *arg)
fprintf(stderr, "client %d aborted in state %d; execution of meta-command failed\n",
i, st->state);
remains--; /* I've aborted */
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
}
}
@@ -4460,9 +4524,21 @@ done:
}
fclose(thread->logfile);
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index a152371e61..83bfbfac47 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -393,6 +393,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
pgbench-ppoll.patch (application/octet-stream, 11.2K) download
diff --git a/configure b/configure
index 0d76e5ea42..756371be22 100755
--- a/configure
+++ b/configure
@@ -9511,6 +9511,62 @@ if test "$ac_res" != no; then :
fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing ppoll" >&5
+$as_echo_n "checking for library containing ppoll... " >&6; }
+if ${ac_cv_search_ppoll+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_func_search_save_LIBS=$LIBS
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char ppoll ();
+int
+main ()
+{
+return ppoll ();
+ ;
+ return 0;
+}
+_ACEOF
+for ac_lib in '' c; do
+ if test -z "$ac_lib"; then
+ ac_res="none required"
+ else
+ ac_res=-l$ac_lib
+ LIBS="-l$ac_lib $ac_func_search_save_LIBS"
+ fi
+ if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_search_ppoll=$ac_res
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext
+ if ${ac_cv_search_ppoll+:} false; then :
+ break
+fi
+done
+if ${ac_cv_search_ppoll+:} false; then :
+
+else
+ ac_cv_search_ppoll=no
+fi
+rm conftest.$ac_ext
+LIBS=$ac_func_search_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_ppoll" >&5
+$as_echo "$ac_cv_search_ppoll" >&6; }
+ac_res=$ac_cv_search_ppoll
+if test "$ac_res" != no; then :
+ test "$ac_res" = "none required" || LIBS="$ac_res $LIBS"
+
+fi
+
# Solaris:
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing fdatasync" >&5
$as_echo_n "checking for library containing fdatasync... " >&6; }
@@ -13024,7 +13080,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index bdc41b071f..b73e848732 100644
--- a/configure.in
+++ b/configure.in
@@ -1035,6 +1035,7 @@ AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(shm_open, rt)
AC_SEARCH_LIBS(shm_unlink, rt)
AC_SEARCH_LIBS(clock_gettime, [rt posix4])
+AC_SEARCH_LIBS(ppoll, c)
# Solaris:
AC_SEARCH_LIBS(fdatasync, [rt posix4])
# Required for thread_test.c on Solaris
@@ -1430,7 +1431,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index e37496c971..7968160069 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -56,6 +59,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -90,6 +97,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_CLREV(s) { do { if ((s)->pfdp) { ((s)->pfdp)->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { memset((s)->pfdp, 0, sizeof(struct pollfd)); } } while(0); }
+#define PFD_SETFD(s) { do { ((s)->pfdp)->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_CLREV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -331,6 +359,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* client transaction count, for -t */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -351,6 +380,7 @@ typedef struct
instr_time conn_time;
StatsData stats;
int64 latency_late; /* executed but late transactions */
+ PFD_STRUCT_POLLFD(*pfds)
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -458,6 +488,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -2137,6 +2168,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* Reset session-local state */
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_ZERO(st);
+ PFD_SETFD(st);
}
/*
@@ -2406,8 +2439,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -2444,11 +2476,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
case CSTATE_ABORTED:
case CSTATE_FINISHED:
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -2596,13 +2624,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/* create tables and setup data */
@@ -3767,7 +3789,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0)
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -4463,6 +4489,8 @@ threadRun(void *arg)
}
}
+ PFD_THREAD_INIT(thread, state, nstate);
+
if (!is_connect)
{
/* make connections to the database */
@@ -4470,6 +4498,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4486,14 +4515,17 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
+#ifndef USE_PPOLL
+ fd_set input_mask;
/* identify which client sockets should be checked for input */
FD_ZERO(&input_mask);
maxsock = -1;
+#endif
+
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -4503,8 +4535,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -4542,7 +4573,13 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ maxsock = 0;
+ (st->pfdp)->events = POLL_EVENTS;
+ (st->pfdp)->revents = 0;
+#else
FD_SET(sock, &input_mask);
+#endif
if (maxsock < sock)
maxsock = sock;
}
@@ -4588,14 +4625,26 @@ threadRun(void *arg)
if (min_usec != PG_INT64_MAX)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
if (nsocks < 0)
{
if (errno == EINTR)
@@ -4604,15 +4653,17 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
+#ifndef USE_PPOLL
else
{
/* If we didn't call select(), don't try to read any data */
FD_ZERO(&input_mask);
}
+#endif
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
@@ -4630,8 +4681,18 @@ threadRun(void *arg)
PQerrorMessage(st->con));
goto done;
}
+#ifdef USE_PPOLL
+ if ((st->pfdp)->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x\n",
+ errno, i, ((st->pfdp)->revents & POLL_FAIL));
+ }
+ if (!(st->pfdp)->revents)
+#else
if (!FD_ISSET(sock, &input_mask))
+#endif
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -4645,7 +4706,10 @@ threadRun(void *arg)
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
/* progress report is made by thread 0 for all threads */
@@ -4759,9 +4823,21 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 579d195663..2d016c8a1f 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -390,6 +390,9 @@
/* Define to 1 if you have the <poll.h> header file. */
#undef HAVE_POLL_H
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `posix_fadvise' function. */
#undef HAVE_POSIX_FADVISE
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-09-25 18:07 ` Andres Freund <andres@anarazel.de>
2017-09-25 20:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
3 siblings, 1 reply; 51+ messages in thread
From: Andres Freund @ 2017-09-25 18:07 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
On 2017-09-25 18:01:40 +0000, Rady, Doug wrote:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
Hm, is there any need of using ppoll over poll? IIRC it's a good bit
more common and there's, also iirc, a number of platforms with buggy
ppoll implementations.
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-09-25 18:07 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
@ 2017-09-25 20:40 ` Rady, Doug <radydoug@amazon.com>
0 siblings, 0 replies; 51+ messages in thread
From: Rady, Doug @ 2017-09-25 20:40 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers
On 9/25/17, 11:07, "Andres Freund" <andres@anarazel.de> wrote:
On 2017-09-25 18:01:40 +0000, Rady, Doug wrote:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
Hm, is there any need of using ppoll over poll? IIRC it's a good bit
more common and there's, also iirc, a number of platforms with buggy
ppoll implementations.
Greetings,
Andres Freund
I used ppoll() as it can support the microseconds duration for \SLEEP meta command.
thanks!
doug
--
Doug Rady
Amazon Aurora PostgreSQL
radydoug@amazon.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-09-25 18:10 ` Fabien COELHO <coelho@cri.ensmp.fr>
3 siblings, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2017-09-25 18:10 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
Hello Again,
> Two patches attached.
> One based on REL9_6_STABLE.
I'd be surprise that there would be a backport unless there is a bug, so
this one might not be useful.
> One based on 'master' which can also apply to REL_10_STABLE.
Could you add your patches to the next CF?
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-10-04 06:20 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-23 18:44 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
3 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2017-10-04 06:20 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
>
> One based on 'master' which can also apply to REL_10_STABLE.
/home/fabien/pgbench-ppoll.patch:137: trailing whitespace.
#define PFD_THREAD_INIT(t,s,n) { do ...
error: patch failed: configure:13024
error: configure: patch does not apply
error: patch failed: configure.in:1430
error: configure.in: patch does not apply
error: patch failed: src/bin/pgbench/pgbench.c:4588
error: src/bin/pgbench/pgbench.c: patch does not apply
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-10-04 06:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2017-10-23 18:44 ` Rady, Doug <radydoug@amazon.com>
2017-11-03 05:29 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Rady, Doug @ 2017-10-23 18:44 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers
Fixed the trailing garbage … sigh.
Fixed the init of unused pollfds.
Re-based for both 10 & master.
--
Thank you!
doug
On 10/3/17, 23:21, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
>
> One based on 'master' which can also apply to REL_10_STABLE.
/home/fabien/pgbench-ppoll.patch:137: trailing whitespace.
#define PFD_THREAD_INIT(t,s,n) { do ...
error: patch failed: configure:13024
error: configure: patch does not apply
error: patch failed: configure.in:1430
error: configure.in: patch does not apply
error: patch failed: src/bin/pgbench/pgbench.c:4588
error: src/bin/pgbench/pgbench.c: patch does not apply
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
pgbench11-ppoll-v2.patch (application/octet-stream, 9.0K) download
diff --git a/configure.in b/configure.in
index cea7fd0755..1a163a1be5 100644
--- a/configure.in
+++ b/configure.in
@@ -1004,6 +1004,7 @@ AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(shm_open, rt)
AC_SEARCH_LIBS(shm_unlink, rt)
AC_SEARCH_LIBS(clock_gettime, [rt posix4])
+AC_SEARCH_LIBS(ppoll, c)
# Solaris:
AC_SEARCH_LIBS(fdatasync, [rt posix4])
# Required for thread_test.c on Solaris
@@ -1418,7 +1419,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 5d8a01c72c..342a55baab 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -56,6 +59,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -90,6 +97,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_CLREV(s) { do { if ((s)->pfdp) { (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { (s)->pfdp->fd = 0; (s)->pfdp->events = (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_SETFD(s) { do { (s)->pfdp->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; (s)[_i].pfdp->fd = -1; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_CLREV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -331,6 +359,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* client transaction count, for -t */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -351,6 +380,7 @@ typedef struct
instr_time conn_time;
StatsData stats;
int64 latency_late; /* executed but late transactions */
+ PFD_STRUCT_POLLFD(*pfds)
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -458,6 +488,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -2137,6 +2168,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* Reset session-local state */
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_ZERO(st);
+ PFD_SETFD(st);
}
/*
@@ -2402,8 +2435,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -2441,10 +2473,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
case CSTATE_ABORTED:
case CSTATE_FINISHED:
if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -2592,13 +2621,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/* create tables and setup data */
@@ -3763,7 +3786,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0)
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -4459,6 +4486,8 @@ threadRun(void *arg)
}
}
+ PFD_THREAD_INIT(thread, state, nstate);
+
if (!is_connect)
{
/* make connections to the database */
@@ -4466,6 +4495,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4482,14 +4512,16 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
+#ifndef USE_PPOLL
+ fd_set input_mask;
/* identify which client sockets should be checked for input */
FD_ZERO(&input_mask);
maxsock = -1;
+#endif
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -4499,8 +4531,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -4538,7 +4569,13 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ maxsock = 0;
+ (st->pfdp)->events = POLL_EVENTS;
+ (st->pfdp)->revents = 0;
+#else
FD_SET(sock, &input_mask);
+#endif
if (maxsock < sock)
maxsock = sock;
}
@@ -4586,11 +4623,19 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else /* nothing active, simple sleep */
{
@@ -4599,7 +4644,11 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
}
if (nsocks < 0)
@@ -4610,15 +4659,17 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
+#ifndef USE_PPOLL
else /* min_usec == 0, i.e. something needs to be executed */
{
/* If we didn't call select(), don't try to read any data */
FD_ZERO(&input_mask);
}
+#endif
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
@@ -4637,7 +4688,18 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ if ((st->pfdp)->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x\n",
+ errno, i, ((st->pfdp)->revents & POLL_FAIL));
+ }
+
+ if (!(st->pfdp)->revents)
+#else
if (!FD_ISSET(sock, &input_mask))
+#endif
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -4651,7 +4713,10 @@ threadRun(void *arg)
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
/* progress report is made by thread 0 for all threads */
@@ -4765,9 +4830,21 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index cfdcc5ac62..21d9bc0300 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -394,6 +394,9 @@
/* Define to 1 if you have the <poll.h> header file. */
#undef HAVE_POLL_H
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `posix_fadvise' function. */
#undef HAVE_POSIX_FADVISE
pgbench10-ppoll-v2.patch (application/octet-stream, 9.0K) download
diff --git a/configure.in b/configure.in
index 68a1e0c184..78df83bdd5 100644
--- a/configure.in
+++ b/configure.in
@@ -1035,6 +1035,7 @@ AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(shm_open, rt)
AC_SEARCH_LIBS(shm_unlink, rt)
AC_SEARCH_LIBS(clock_gettime, [rt posix4])
+AC_SEARCH_LIBS(ppoll, c)
# Solaris:
AC_SEARCH_LIBS(fdatasync, [rt posix4])
# Required for thread_test.c on Solaris
@@ -1430,7 +1431,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 2bdfc89d2a..79bbe3ec7a 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -56,6 +59,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -90,6 +97,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_CLREV(s) { do { if ((s)->pfdp) { (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { (s)->pfdp->fd = 0; (s)->pfdp->events = (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_SETFD(s) { do { (s)->pfdp->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; (s)[_i].pfdp->fd = -1; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_CLREV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -331,6 +359,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* transaction count */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -351,6 +380,7 @@ typedef struct
instr_time conn_time;
StatsData stats;
int64 latency_late; /* executed but late transactions */
+ PFD_STRUCT_POLLFD(*pfds)
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -458,6 +488,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -2100,6 +2131,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* Reset session-local state */
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_ZERO(st);
+ PFD_SETFD(st);
}
/*
@@ -2372,8 +2405,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -2412,10 +2444,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
case CSTATE_ABORTED:
case CSTATE_FINISHED:
if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -2553,13 +2582,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/* create tables and setup data */
@@ -3730,7 +3753,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0)
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -4420,6 +4447,8 @@ threadRun(void *arg)
}
}
+ PFD_THREAD_INIT(thread, state, nstate);
+
if (!is_connect)
{
/* make connections to the database */
@@ -4427,6 +4456,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4443,14 +4473,16 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
+#ifndef USE_PPOLL
+ fd_set input_mask;
/* identify which client sockets should be checked for input */
FD_ZERO(&input_mask);
maxsock = -1;
+#endif
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -4460,8 +4492,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -4499,7 +4530,13 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ maxsock = 0;
+ (st->pfdp)->events = POLL_EVENTS;
+ (st->pfdp)->revents = 0;
+#else
FD_SET(sock, &input_mask);
+#endif
if (maxsock < sock)
maxsock = sock;
}
@@ -4547,11 +4584,19 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else /* nothing active, simple sleep */
{
@@ -4560,7 +4605,11 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
}
if (nsocks < 0)
@@ -4571,15 +4620,17 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
+#ifndef USE_PPOLL
else /* min_usec == 0, i.e. something needs to be executed */
{
/* If we didn't call select(), don't try to read any data */
FD_ZERO(&input_mask);
}
+#endif
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
@@ -4598,7 +4649,18 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ if ((st->pfdp)->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x\n",
+ errno, i, ((st->pfdp)->revents & POLL_FAIL));
+ }
+
+ if (!(st->pfdp)->revents)
+#else
if (!FD_ISSET(sock, &input_mask))
+#endif
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -4612,7 +4674,10 @@ threadRun(void *arg)
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
/* progress report is made by thread 0 for all threads */
@@ -4726,9 +4791,21 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index c65dd7db21..63c0e74ff7 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -390,6 +390,9 @@
/* Define to 1 if you have the <poll.h> header file. */
#undef HAVE_POLL_H
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `posix_fadvise' function. */
#undef HAVE_POSIX_FADVISE
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-10-04 06:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-23 18:44 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-11-03 05:29 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-28 01:41 ` Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts Michael Paquier <michael.paquier@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2017-11-03 05:29 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
Hello,
Could you rebase the v11 patch?
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-10-04 06:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-23 18:44 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-03 05:29 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2017-11-28 01:41 ` Michael Paquier <michael.paquier@gmail.com>
2017-11-28 05:39 ` Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Michael Paquier @ 2017-11-28 01:41 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
On Fri, Nov 3, 2017 at 2:29 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
> Could you rebase the v11 patch?
This patch has been waiting for a rebase for more than three weeks as
of today, I am marking it as returned with feedback. It would be a
good idea to reply to Robert's input in
https://www.postgresql.org/message-id/CA+TgmoYybNv-DdhVPMxLB2nx2SqeNJirtWHmdEAZUCGoTB2VBg@mail.gmail....
--
Michael
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-10-04 06:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-23 18:44 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-03 05:29 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-28 01:41 ` Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts Michael Paquier <michael.paquier@gmail.com>
@ 2017-11-28 05:39 ` Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2017-11-28 05:39 UTC (permalink / raw)
To: Michael Paquier <michael.paquier@gmail.com>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
Hello Michaël,
>> Could you rebase the v11 patch?
>
> This patch has been waiting for a rebase for more than three weeks as
> of today, I am marking it as returned with feedback. It would be a
> good idea to reply to Robert's input in
> https://www.postgresql.org/message-id/CA+TgmoYybNv-DdhVPMxLB2nx2SqeNJirtWHmdEAZUCGoTB2VBg@mail.gmail....
ISTM that this was done: If -c is high enough, pgbench fails without the
patch.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-10-26 11:45 ` Robert Haas <robertmhaas@gmail.com>
2017-10-26 13:21 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
3 siblings, 1 reply; 51+ messages in thread
From: Robert Haas @ 2017-10-26 11:45 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
On Mon, Sep 25, 2017 at 8:01 PM, Rady, Doug <radydoug@amazon.com> wrote:
> This patch enables building pgbench to use ppoll() instead of select()
>
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
>
> when using ppoll(), the only connection limitation is system resources.
So what's an example of something that fails without this patch but
works with the patch?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-10-26 11:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
@ 2017-10-26 13:21 ` Rady, Doug <radydoug@amazon.com>
0 siblings, 0 replies; 51+ messages in thread
From: Rady, Doug @ 2017-10-26 13:21 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers
Without this patch, one is limited to '(FD_SETSIZE - 10)’ number of connections.
Example of something that fails without this patch but works with the patch:
Without the patch:
$ pgbench -j 3000 -c 1500
invalid number of clients: "1500"
With the patch:
$ pgbench -j 3000 -c 1500
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 2000
query mode: simple
number of clients: 1500
number of threads: 1500
number of transactions per client: 10
number of transactions actually processed: 15000/15000
latency average = 631.730 ms
tps = 2374.430587 (including connections establishing)
tps = 4206.524986 (excluding connections establishing)
--
doug
On 10/26/17, 04:46, "Robert Haas" <robertmhaas@gmail.com> wrote:
On Mon, Sep 25, 2017 at 8:01 PM, Rady, Doug <radydoug@amazon.com> wrote:
> This patch enables building pgbench to use ppoll() instead of select()
>
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
>
> when using ppoll(), the only connection limitation is system resources.
So what's an example of something that fails without this patch but
works with the patch?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 51+ messages in thread
* PATCH: pgbench - option to build using ppoll() for larger connection counts
@ 2017-09-25 22:29 Rady, Doug <radydoug@amazon.com>
0 siblings, 0 replies; 51+ messages in thread
From: Rady, Doug @ 2017-09-25 22:29 UTC (permalink / raw)
To: pgsql-hackers
This patch enables building pgbench to use ppoll() instead of select()
to allow for more than (FD_SETSIZE - 10) connections. As implemented,
when using ppoll(), the only connection limitation is system resources.
To maintain the microseconds duration of \SLEEP meta command, ppoll() was used instead of poll().
The patch has been implemented to introduce a minimal of #ifdef/#ifndef
clutter in the code.
(this is just a re-send w/o the 9.6 patch as commitfest latched onto the 9.6 patch)
doug
--
Doug Rady
Amazon Aurora PostgreSQL
radydoug@amazon.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
pgbench-ppoll-v1.patch (application/octet-stream, 11.2K) download
diff --git a/configure b/configure
index 0d76e5ea42..756371be22 100755
--- a/configure
+++ b/configure
@@ -9511,6 +9511,62 @@ if test "$ac_res" != no; then :
fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing ppoll" >&5
+$as_echo_n "checking for library containing ppoll... " >&6; }
+if ${ac_cv_search_ppoll+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_func_search_save_LIBS=$LIBS
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char ppoll ();
+int
+main ()
+{
+return ppoll ();
+ ;
+ return 0;
+}
+_ACEOF
+for ac_lib in '' c; do
+ if test -z "$ac_lib"; then
+ ac_res="none required"
+ else
+ ac_res=-l$ac_lib
+ LIBS="-l$ac_lib $ac_func_search_save_LIBS"
+ fi
+ if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_search_ppoll=$ac_res
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext
+ if ${ac_cv_search_ppoll+:} false; then :
+ break
+fi
+done
+if ${ac_cv_search_ppoll+:} false; then :
+
+else
+ ac_cv_search_ppoll=no
+fi
+rm conftest.$ac_ext
+LIBS=$ac_func_search_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_ppoll" >&5
+$as_echo "$ac_cv_search_ppoll" >&6; }
+ac_res=$ac_cv_search_ppoll
+if test "$ac_res" != no; then :
+ test "$ac_res" = "none required" || LIBS="$ac_res $LIBS"
+
+fi
+
# Solaris:
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing fdatasync" >&5
$as_echo_n "checking for library containing fdatasync... " >&6; }
@@ -13024,7 +13080,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index bdc41b071f..b73e848732 100644
--- a/configure.in
+++ b/configure.in
@@ -1035,6 +1035,7 @@ AC_SEARCH_LIBS(crypt, crypt)
AC_SEARCH_LIBS(shm_open, rt)
AC_SEARCH_LIBS(shm_unlink, rt)
AC_SEARCH_LIBS(clock_gettime, [rt posix4])
+AC_SEARCH_LIBS(ppoll, c)
# Solaris:
AC_SEARCH_LIBS(fdatasync, [rt posix4])
# Required for thread_test.c on Solaris
@@ -1430,7 +1431,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range towlower utime utimes wcstombs wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index e37496c971..7968160069 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -56,6 +59,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -90,6 +97,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_CLREV(s) { do { if ((s)->pfdp) { ((s)->pfdp)->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { memset((s)->pfdp, 0, sizeof(struct pollfd)); } } while(0); }
+#define PFD_SETFD(s) { do { ((s)->pfdp)->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_CLREV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -331,6 +359,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* client transaction count, for -t */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -351,6 +380,7 @@ typedef struct
instr_time conn_time;
StatsData stats;
int64 latency_late; /* executed but late transactions */
+ PFD_STRUCT_POLLFD(*pfds)
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -458,6 +488,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -2137,6 +2168,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* Reset session-local state */
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_ZERO(st);
+ PFD_SETFD(st);
}
/*
@@ -2406,8 +2439,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -2444,11 +2476,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
case CSTATE_ABORTED:
case CSTATE_FINISHED:
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -2596,13 +2624,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/* create tables and setup data */
@@ -3767,7 +3789,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0)
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -4463,6 +4489,8 @@ threadRun(void *arg)
}
}
+ PFD_THREAD_INIT(thread, state, nstate);
+
if (!is_connect)
{
/* make connections to the database */
@@ -4470,6 +4498,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4486,14 +4515,17 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
+#ifndef USE_PPOLL
+ fd_set input_mask;
/* identify which client sockets should be checked for input */
FD_ZERO(&input_mask);
maxsock = -1;
+#endif
+
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -4503,8 +4535,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -4542,7 +4573,13 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ maxsock = 0;
+ (st->pfdp)->events = POLL_EVENTS;
+ (st->pfdp)->revents = 0;
+#else
FD_SET(sock, &input_mask);
+#endif
if (maxsock < sock)
maxsock = sock;
}
@@ -4588,14 +4625,26 @@ threadRun(void *arg)
if (min_usec != PG_INT64_MAX)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
if (nsocks < 0)
{
if (errno == EINTR)
@@ -4604,15 +4653,17 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
+#ifndef USE_PPOLL
else
{
/* If we didn't call select(), don't try to read any data */
FD_ZERO(&input_mask);
}
+#endif
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
@@ -4630,8 +4681,18 @@ threadRun(void *arg)
PQerrorMessage(st->con));
goto done;
}
+#ifdef USE_PPOLL
+ if ((st->pfdp)->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x\n",
+ errno, i, ((st->pfdp)->revents & POLL_FAIL));
+ }
+ if (!(st->pfdp)->revents)
+#else
if (!FD_ISSET(sock, &input_mask))
+#endif
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -4645,7 +4706,10 @@ threadRun(void *arg)
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
/* progress report is made by thread 0 for all threads */
@@ -4759,9 +4823,21 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 579d195663..2d016c8a1f 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -390,6 +390,9 @@
/* Define to 1 if you have the <poll.h> header file. */
#undef HAVE_POLL_H
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `posix_fadvise' function. */
#undef HAVE_POSIX_FADVISE
* PATCH: pgbench - option to build using ppoll() for larger connection counts
@ 2017-11-28 15:38 Rady, Doug <radydoug@amazon.com>
2017-11-28 18:05 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
2017-11-29 08:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-06 21:49 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Stephen Frost <sfrost@snowman.net>
0 siblings, 3 replies; 51+ messages in thread
From: Rady, Doug @ 2017-11-28 15:38 UTC (permalink / raw)
To: pgsql-hackers
[trying again for 2018-01]
This patch enables building pgbench to use ppoll() instead of select()
to allow for more than (FD_SETSIZE - 10) connections. As implemented,
when using ppoll(), the only connection limitation is system resources.
The patch has been implemented to introduce a minimal of #ifdef/#ifndef
clutter in the code.
Without this patch, one is limited to '(FD_SETSIZE - 10)’ number of connections.
Example of something that fails without this patch but works with the patch:
Without the patch:
$ pgbench -j 3000 -c 1500
invalid number of clients: "1500"
With the patch:
$ pgbench -j 3000 -c 1500
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 2000
query mode: simple
number of clients: 1500
number of threads: 1500
number of transactions per client: 10
number of transactions actually processed: 15000/15000
latency average = 631.730 ms
tps = 2374.430587 (including connections establishing)
tps = 4206.524986 (excluding connections establishing)
doug
--
Doug Rady
Amazon Aurora, RDS PostgreSQL
radydoug@amazon.com
Attachments:
pgbench11-ppoll-v3.patch (application/octet-stream, 9.8K) download
diff --git a/configure.in b/configure.in
index d9c4a50b4b..27cd84d135 100644
--- a/configure.in
+++ b/configure.in
@@ -1424,7 +1424,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index bd96eae5e6..09c3e60ffb 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -56,6 +59,10 @@
#define M_PI 3.14159265358979323846
#endif
+#ifdef HAVE_PPOLL
+#define USE_PPOLL 1
+#endif
+
#include "pgbench.h"
#define ERRCODE_UNDEFINED_TABLE "42P01"
@@ -90,6 +97,27 @@ static int pthread_join(pthread_t th, void **thread_return);
#define MAXCLIENTS 1024
#endif
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_RESETEV(s) { do { if ((s)->pfdp) { (s)->pfdp->events = POLL_EVENTS; (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { (s)->pfdp->fd = -1; (s)->pfdp->events = POLL_EVENTS; (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_SETFD(s) { do { (s)->pfdp->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) { pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i < (n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; (s)[_i].pfdp->fd = -1; } } while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_RESETEV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
+
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
@@ -328,6 +356,7 @@ typedef struct
/* per client collected stats */
int64 cnt; /* client transaction count, for -t */
int ecnt; /* error count */
+ PFD_STRUCT_POLLFD(*pfdp)
} CState;
/*
@@ -348,6 +377,7 @@ typedef struct
instr_time conn_time;
StatsData stats;
int64 latency_late; /* executed but late transactions */
+ PFD_STRUCT_POLLFD(*pfds)
} TState;
#define INVALID_THREAD ((pthread_t) 0)
@@ -465,6 +495,7 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
/* callback functions for our flex lexer */
@@ -933,6 +964,7 @@ discard_response(CState *state)
if (res)
PQclear(res);
} while (res);
+ PFD_RESETEV(state);
}
/* qsort comparator for Variable array */
@@ -2157,6 +2189,10 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+ if (debug)
+ fprintf(stderr, "client %d connecting ...\n",
+ st->id);
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -2169,6 +2205,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* Reset session-local state */
memset(st->prepared, 0, sizeof(st->prepared));
+ PFD_ZERO(st);
+ PFD_SETFD(st);
}
/*
@@ -2434,8 +2472,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -2472,11 +2509,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
case CSTATE_ABORTED:
case CSTATE_FINISHED:
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -2624,13 +2657,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/*
@@ -3949,7 +3976,11 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
+#ifdef USE_PPOLL
+ if (nclients <= 0)
+#else
if (nclients <= 0 || nclients > MAXCLIENTS)
+#endif
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -4629,6 +4660,8 @@ threadRun(void *arg)
StatsData last,
aggs;
+ PFD_THREAD_INIT(thread, state, nstate);
+
/*
* Initialize throttling rate target for all of the thread's clients. It
* might be a little more accurate to reset thread->start_time here too.
@@ -4669,8 +4702,12 @@ threadRun(void *arg)
/* make connections to the database */
for (i = 0; i < nstate; i++)
{
+ if (debug)
+ fprintf(stderr, "client %d connecting\n",
+ state[i].id);
if ((state[i].con = doConnect()) == NULL)
goto done;
+ PFD_SETFD(&state[i]);
}
}
@@ -4687,13 +4724,15 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
+#ifndef USE_PPOLL
+ fd_set input_mask;
/* identify which client sockets should be checked for input */
FD_ZERO(&input_mask);
+#endif
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -4704,8 +4743,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -4743,7 +4781,11 @@ threadRun(void *arg)
goto done;
}
+#ifdef USE_PPOLL
+ PFD_RESETEV(st);
+#else
FD_SET(sock, &input_mask);
+#endif
if (maxsock < sock)
maxsock = sock;
}
@@ -4791,11 +4833,19 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
+#ifdef USE_PPOLL
+ struct timespec timeout;
+
+ timeout.tv_sec = min_usec / 1000000;
+ timeout.tv_nsec = min_usec % 1000000000;
+ nsocks = ppoll(thread->pfds, nstate, &timeout, NULL);
+#else
struct timeval timeout;
timeout.tv_sec = min_usec / 1000000;
timeout.tv_usec = min_usec % 1000000;
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+#endif
}
else /* nothing active, simple sleep */
{
@@ -4804,7 +4854,11 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
+#ifdef USE_PPOLL
+ nsocks = ppoll(thread->pfds, nstate, NULL, NULL);
+#else
nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+#endif
}
if (nsocks < 0)
@@ -4815,10 +4869,11 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SCKTWTMTHD, strerror(errno));
goto done;
}
}
+#ifndef USE_PPOLL
else
{
/* min_usec == 0, i.e. something needs to be executed */
@@ -4826,6 +4881,7 @@ threadRun(void *arg)
/* If we didn't call select(), don't try to read any data */
FD_ZERO(&input_mask);
}
+#endif
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
@@ -4837,14 +4893,30 @@ threadRun(void *arg)
/* don't call doCustom unless data is available */
int sock = PQsocket(st->con);
+#ifdef USE_PPOLL
+ if (sock < 0 && st->pfdp->revents & POLL_FAIL)
+ {
+ fprintf(stderr,
+ "ppoll() fail - errno: %d, i: %d, events: %x, %s\n",
++ errno, i,
+ ((st->pfdp)->revents & POLL_FAIL),
+ PQerrorMessage(st->con));
+ goto done;
+ }
+#else
if (sock < 0)
{
fprintf(stderr, "invalid socket: %s",
PQerrorMessage(st->con));
goto done;
}
+#endif
+#ifdef USE_PPOLL
+ if (!(st->pfdp)->revents)
+#else
if (!FD_ISSET(sock, &input_mask))
+#endif
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -4858,7 +4930,10 @@ threadRun(void *arg)
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
+ {
remains--;
+ PFD_ZERO(st);
+ }
}
/* progress report is made by thread 0 for all threads */
@@ -4982,9 +5057,21 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ PFD_THREAD_FREE(thread);
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+ PFD_ZERO(st);
+}
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 84d59f12b2..21e720cfab 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -406,6 +406,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-11-28 18:05 ` Robert Haas <robertmhaas@gmail.com>
2 siblings, 0 replies; 51+ messages in thread
From: Robert Haas @ 2017-11-28 18:05 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
On Tue, Nov 28, 2017 at 10:38 AM, Rady, Doug <radydoug@amazon.com> wrote:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
IIUC, ppoll() is to poll() as pselect() is to select(). Since the
existing code uses select(), why not convert to poll() rather than
ppoll()?
+#ifdef USE_PPOLL
+#define SCKTWTMTHD "ppoll"
+#undef MAXCLIENTS
+#define POLL_EVENTS (POLLIN|POLLPRI|POLLRDHUP)
+#define POLL_FAIL (POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#define PFD_RESETEV(s) { do { if ((s)->pfdp) { (s)->pfdp->events =
POLL_EVENTS; (s)->pfdp->revents = 0; } } while(0); }
+#define PFD_ZERO(s) { do { if ((s)->pfdp) { (s)->pfdp->fd = -1;
(s)->pfdp->events = POLL_EVENTS; (s)->pfdp->revents = 0; } } while(0);
}
+#define PFD_SETFD(s) { do { (s)->pfdp->fd = PQsocket((s)->con); } while(0); }
+#define PFD_STRUCT_POLLFD(p) struct pollfd (p);
+#define PFD_THREAD_FREE(t) { do { if ((t)->pfds) {
pg_free((t)->pfds); (t)->pfds = NULL; } } while (0); }
+#define PFD_THREAD_INIT(t,s,n) { do { int _i; (t)->pfds = (struct
pollfd *) pg_malloc0(sizeof(struct pollfd) * (n)); for (_i = 0; _i <
(n); _i++) { (s)[_i].pfdp = &(t)->pfds[_i]; (s)[_i].pfdp->fd = -1; } }
while (0); }
+#else
+#define SCKTWTMTHD "select"
+#define PFD_RESETEV(s)
+#define PFD_ZERO(s)
+#define PFD_SETFD(s)
+#define PFD_STRUCT_POLLFD(p)
+#define PFD_THREAD_FREE(t)
+#define PFD_THREAD_INIT(t,s,n)
+#endif
I find this an impenetrable mess. I am not going to commit a macro
with a ten-letter name that contains neither punctuation marks nor
vowels, nor one that is a single 200+ character that embeds a malloc
and a loop call but no comments. While I agree with the general goal
of avoiding introducing lots of #ifdefs, and while I think that the
introduction of finishCon() seems like a possibly-useful step in that
direction, I don't think that having a bunch of long, complex,
comment-free macros like this actually improves code clarity. Better
to have some embedded #ifdefs than this.
+ if (debug)
+ fprintf(stderr, "client %d connecting ...\n",
+ st->id);
+
This is unrelated to the purpose of the patch. Please don't mix in
unrelated changes.
+ if (debug)
+ fprintf(stderr, "client %d connecting\n",
+ state[i].id);
Ditto.
IMHO, this looks like a broadly reasonable thing to do. I had no idea
that FD_SETSIZE was ever set to a value much less than the number of
FDs the system could handle -- but if it is, then this seems like a
reasonable fix, once we agree on exactly how the details should look.
I don't remember off-hand whether you've done testing to see how
poll()/ppoll() performs compares to select(), but that might also be
something to check.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2017-11-29 08:40 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 12:58 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
2 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2017-11-29 08:40 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
Hello,
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
I'm fine with allowing more clients through ppoll, as large
multi/many/whatever-core hardware becomes more common and is expected to
grow.
However, I'm at odds with the how and the ppoll/select compatibility layer
offered by the patch, and the implied maintenance cost just to understand
what is happening while reading the code cluttered with possibly empty
macros and ifdef stuff. There are many ifdef, many strange/astute macros,
significant dissymetry in the code, which reduce maintainability
noticeably.
I think that it should abstract cleanly the two implementations into a set
of functions with the same signatures, even if some arguments are unsused,
and use these functions without ifdef stuff later. Maybe some macros too,
ok, but not too much.
"SCKTWTMTHD"... WTH? I do not want to know what it means. I'm sure a
better name (if something without vowels can be a name of anything in
English...) can be found. Names must be chosen with great care.
MAXCLIENTS: could be set to -1 when there is no limit, and tested to avoid
one ifdef.
I must admit that I do not see the benefit of "{ do { ... } while(0); }"
compared to "{ ... }". Maybe it has to do with compiler warnings.
> The patch has been implemented to introduce a minimal of #ifdef/#ifndef
> clutter in the code.
I think that it could be much more minimal than that. I would aim at
having just one.
#ifdef USE_PPOLL
specific functions definitions
some macros
#else /* SELECT */
idem for select
#endif
Ok, it may not be the best solution everywhere, but it should be
significantly reduce compared to the current state.
ISTM that ifdef which breaks the code structure should be avoided, eg
#ifdef ...
if (...)
#else
if (...)
#endif
{
// condition was true...
It is unclear to me why the input_mask is declared in the threadRun
function (so is thread specific) but pfds is in each thread struct (so is
also thread specific, but differently). The same logic should be used for
both, which ever is best. Not sure that "pfds" is the right name. If the
two variables means the same thing, they should have the same name,
although possibly different types.
> $ pgbench -j 3000 -c 1500
Probably you intended 1500 threads for 3000 clients.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-29 08:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2017-11-29 12:58 ` Robert Haas <robertmhaas@gmail.com>
2017-11-29 13:10 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Robert Haas @ 2017-11-29 12:58 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
On Wed, Nov 29, 2017 at 3:40 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
> ever is best. Not sure that "pfds" is the right name. If the two variables
> means the same thing, they should have the same name, although possibly
> different types.
Although I agree with a good bit of what you say here, I don't agree
with that. If the member used by ppoll() (or just poll()) has a
different name than the one used for select(), it's much easier to,
say, grep for everyplace where the field you care about is used. If
you use the same name for different things, that doesn't work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-29 08:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 12:58 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
@ 2017-11-29 13:10 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 14:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2017-11-29 13:10 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
Hello Robert,
>> ever is best. Not sure that "pfds" is the right name. If the two variables
>> means the same thing, they should have the same name, although possibly
>> different types.
>
> Although I agree with a good bit of what you say here, I don't agree
> with that. If the member used by ppoll() (or just poll()) has a
> different name than the one used for select(), it's much easier to,
> say, grep for everyplace where the field you care about is used. If
> you use the same name for different things, that doesn't work.
I'm not sure it is incompatible.
My point is consistent with my other advice which is to hide the stuff in
functions with identical (or compatible) signatures, so that the only
place where it would differ would be in the functions, where greping would
work.
#ifdef USE_POLL
// pool specific stuff
#define SOME_TYPE v1_type (or typedef)
void do_stuff(v1_type * stuff) { ... }
...
#else /* default to SELECT */
// select specific stuff
#define SOME_TYPE v2_type (idem)
void do_stuff(v2_type * stuff) { ... }
...
#endif
Then later the code is not specific to poll or select, eg:
SOME_TYPE mask;
do_stuff(mask);
do_other_stuff(...);
if (is_ready(mask, ...)) { ... }
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-29 08:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 12:58 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
2017-11-29 13:10 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2017-11-29 14:15 ` Robert Haas <robertmhaas@gmail.com>
2017-11-29 14:26 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Robert Haas @ 2017-11-29 14:15 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
On Wed, Nov 29, 2017 at 8:10 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
> My point is consistent with my other advice which is to hide the stuff in
> functions with identical (or compatible) signatures, so that the only place
> where it would differ would be in the functions, where greping would work.
>
> #ifdef USE_POLL
> // pool specific stuff
> #define SOME_TYPE v1_type (or typedef)
> void do_stuff(v1_type * stuff) { ... }
> ...
> #else /* default to SELECT */
> // select specific stuff
> #define SOME_TYPE v2_type (idem)
> void do_stuff(v2_type * stuff) { ... }
> ...
> #endif
>
> Then later the code is not specific to poll or select, eg:
>
> SOME_TYPE mask;
> do_stuff(mask);
> do_other_stuff(...);
> if (is_ready(mask, ...)) { ... }
Yeah, that sort of style would be OK with me. But I wouldn't like:
struct blah {
#ifdef FOO
int doohicky;
#else
char *doohicky;
};
...because now any place in the code where you see "doohicky" you
don't immediately know whether it's an int or a char *
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-29 08:40 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 12:58 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
2017-11-29 13:10 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 14:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Robert Haas <robertmhaas@gmail.com>
@ 2017-11-29 14:26 ` Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2017-11-29 14:26 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers
> [...] Yeah, that sort of style would be OK with me. But I wouldn't
> like:
>
> struct blah {
> #ifdef FOO
> int doohicky;
> #else
> char *doohicky;
> };
Indeed. Me neither.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-06 21:49 ` Stephen Frost <sfrost@snowman.net>
2 siblings, 0 replies; 51+ messages in thread
From: Stephen Frost @ 2018-01-06 21:49 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers
Doug,
* Rady, Doug (radydoug@amazon.com) wrote:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
Looks like this patch had some good feedback back when it was posted,
but we haven't seen any update to it based on that feedback and we're
now nearly a week into the January commitfest.
In order to keep the commitfest moving and to give you the best
opportunity at having the patch be reviewed and committed during this
cycle, I'd suggest trying to find some time soon to incorporate the
recommendations provided and post an updated patch for review.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 51+ messages in thread
* PATCH: pgbench - option to build using ppoll() for larger connection counts
@ 2018-01-22 08:34 Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-09-19 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 51+ messages in thread
From: Rady, Doug @ 2018-01-22 08:34 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Robert Haas <robertmhaas@gmail.com>
This version of the patch attempts to address the feedback for the previous
submission on 28-Nov-2017
This patch enables building pgbench to use ppoll() instead of select()
to allow for more than (FD_SETSIZE - 10) connections. As implemented,
when using ppoll(), the only connection limitation is system resources.
“… ppoll() is to poll() as pselect() is to select(). Since the
existing code uses select(), why not convert to poll() rather than
ppoll()?”
Time in pgbench is measured in microseconds.
The select() uses microseconds.
The ppoll() and pselect() call use nanoseconds
The poll() call uses milliseconds.
In order to not downgrade time resolution, ppoll() is used instead of poll().
Without this patch, one is limited to '(FD_SETSIZE - 10)’ number of connections.
Example of something that fails without this patch but works with the patch:
Without the patch:
$ pgbench -j 1500 -c 1500
invalid number of clients: "1500"
With the patch:
$ pgbench -j 1500 -c 1500
starting vacuum...end.
progress: 12.0 s, 782.3 tps, lat 917.507 ms stddev 846.929
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 2000
query mode: simple
number of clients: 1500
number of threads: 1500
number of transactions per client: 10
number of transactions actually processed: 15000/15000
latency average = 1180.216 ms
latency stddev = 855.126 ms
tps = 658.674816 (including connections establishing)
tps = 765.289160 (excluding connections establishing)
--
Doug Rady
Amazon Aurora, RDS PostgreSQL
radydoug@amazon.com
Attachments:
pgbench11-ppoll-v6.patch (application/octet-stream, 10.7K) download
diff --git a/configure.in b/configure.in
index 4d26034579..7b2b531675 100644
--- a/configure.in
+++ b/configure.in
@@ -1425,7 +1425,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 31ea6ca06e..05773d8bb1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -83,12 +86,28 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-/* max number of clients allowed */
+#if defined(HAVE_PPOLL)
+#define USE_PPOLL 1
+#endif
+
+#if defined(USE_PPOLL)
+/* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_array;
+/* unlimited number of clients */
+#define MAXCLIENTS -1
+#else
+/* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_array;
#ifdef FD_SETSIZE
+/* system limited max number of clients allowed */
#define MAXCLIENTS (FD_SETSIZE - 10)
#else
+/* max number of clients allowed */
#define MAXCLIENTS 1024
#endif
+#endif
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -502,6 +521,15 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
+static socket_array *alloc_socket_array(int count);
+static void clear_socket(socket_array *sa, int fd, int idx);
+static int error_on_socket(socket_array *sa, int idx, PGconn *con);
+static void free_socket_array(socket_array *sa);
+static int ignore_socket(socket_array *sa, int idx, PGconn *con);
+static void init_socket_array(socket_array *sa, int count);
+static void set_socket(socket_array *sa, int fd, int idx);
+static int wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -2704,6 +2732,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -2981,8 +3010,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -3019,11 +3047,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
case CSTATE_ABORTED:
case CSTATE_FINISHED:
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -3171,13 +3195,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/*
@@ -4507,7 +4525,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5189,6 +5207,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_array *sockets = alloc_socket_array(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5232,6 +5251,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5248,13 +5268,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
- /* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ init_socket_array(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5265,8 +5284,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -5304,7 +5322,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5341,7 +5359,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5352,11 +5370,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_array(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5365,7 +5379,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_array(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5376,7 +5390,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5385,7 +5399,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ init_socket_array(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5396,16 +5410,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5543,9 +5552,137 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_array(sockets);
+ sockets = NULL;
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+}
+
+static socket_array *
+alloc_socket_array(int count)
+{
+ return (socket_array *) pg_malloc0(sizeof(socket_array) * ((MAXCLIENTS > 0) ? 1 : count));
+}
+
+static void
+free_socket_array(socket_array *sa)
+{
+ pg_free(sa);
+}
+
+#if defined(USE_PPOLL)
+#ifdef POLLRDHUP
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+#define POLL_ISSUES (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#else
+#define POLL_EVENTS (POLLIN|POLLPRI)
+#define POLL_ISSUES (POLLERR|POLLHUP|POLLNVAL)
+#endif
+
+static void
+clear_socket(socket_array *sa, int fd, int idx)
+{
+ set_socket(sa, -1, idx);
+}
+
+static int
+error_on_socket(socket_array *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_ISSUES)))
+ return 0;
+ fprintf(stderr, "ppoll() fail - errno: %d, idx: %d, events: %x, %s\n",
+ errno, idx, (sa[idx].revents & POLL_ISSUES),
+ PQerrorMessage(con));
+ return 1;
+}
+
+static int
+ignore_socket(socket_array *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents) ? 1 : 0;
+}
+
+static void
+init_socket_array(socket_array *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ clear_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_array *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ }
+ return ppoll(sa, nstate, usec ? &timeout : NULL, NULL);
+}
+#else
+static void
+clear_socket(socket_array *sa, int fd, int idx)
+{
+ FD_CLR(fd, sa);
+}
+
+static int
+error_on_socket(socket_array *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return 0;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return 0;
+}
+
+static int
+ignore_socket(socket_array *sa, int idx, PGconn *con)
+{
+ return (!FD_ISSET(PQsocket(con), sa)) ? 1 : 0;
+}
+
+static void
+init_socket_array(socket_array *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_array *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, usec ? &timeout : NULL);
+}
+#endif
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f98f773ff0..63207e13fd 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -409,6 +409,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-24 08:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:05 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
1 sibling, 2 replies; 51+ messages in thread
From: Fabien COELHO @ 2018-01-24 08:00 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
> This version of the patch attempts to address the feedback for the
> previous submission on 28-Nov-2017
Please avoid recreating a thread, but rather respond to the previous one,
I missed this post.
The overall function-based implementation with limited ifdefs seems
readable and maintainable. I think that it rather improves the code in
places by hiding details.
Patch applies with one warning:
pgbench11-ppoll-v6.patch:141: trailing whitespace.
set_socket(sockets, PQsocket(state[i].con), i);<SPACE>
warning: 1 line adds whitespace errors.
The patch implies to run "autoconf" to regenerate the configure script.
Compilation with "ppoll" ok, globale & local make check ok. I do not have
hardware which allows to run with over 1024 clients, so I cannot say that
I tested the case.
Compilation without "ppoll" gets a warning:
pgbench.c:5645:1: warning: ‘clear_socket’ defined but not used...
clear_socket(socket_array *sa, int fd, int idx)
The "clear_socket" function is not used in this case. I'd suggest to
remove it from the prototypes, remove or comment the unused implementation
in the select section, and keep the one in the ppoll section. Or use it if
it is needed. Or inline it in the "init_socket_array" function where it is
used just once.
I'm not sure of the name "socket_array", because it is an array only in
the ppoll case. Maybe "sockets_t" or "socket_set"? Or something else?
Maybe "init_socket_array" should be named "clear_...".
I would suggest not to include sys/select.h when using ppoll, as it is a useless
include this case. I.e. move includes in the ifdef USE_PPOLL section?
Please do not remove comments, eg:
- /* identify which client sockets should be checked for input */
On #endif or #else, especially large scope ones, I would have a comment to
say about what it is, eg at the minimum:
#else /* select(2) implementation */
#endif /* USE_PPOLL */
On:
+#if defined(USE_PPOLL)
+#ifdef POLLRDHUP
Use just one "ifdef" style?
There should be a comment about what this sections are providing, eg:
/* ppoll(2) implementation for "socket_array" functions */
There should be an empty line and possibly a comment explaining why
POLLRDHUP may not be there and/or why this define is necessary.
With select you always initialize the timeout, but not with ppoll.
Use a common approach in the implementations?
The "finishCon" function addition seems totally unrelated to this patch.
Although I agree that this function improves the code, it is refactoring
and does not really belong here.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-01-25 15:05 ` Rady, Doug <radydoug@amazon.com>
2018-01-25 15:16 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
1 sibling, 1 reply; 51+ messages in thread
From: Rady, Doug @ 2018-01-25 15:05 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 1/24/18, 00:00, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
Hello Doug,
Hello Fabien,
> This version of the patch attempts to address the feedback for the
> previous submission on 28-Nov-2017
Please avoid recreating a thread, but rather respond to the previous one,
I missed this post.
Got it.
The overall function-based implementation with limited ifdefs seems
readable and maintainable. I think that it rather improves the code in
places by hiding details.
Patch applies with one warning:
pgbench11-ppoll-v6.patch:141: trailing whitespace.
set_socket(sockets, PQsocket(state[i].con), i);<SPACE>
warning: 1 line adds whitespace errors.
Fixed.
The patch implies to run "autoconf" to regenerate the configure script.
Yes. I should have included this in the description.
Compilation with "ppoll" ok, globale & local make check ok. I do not have
hardware which allows to run with over 1024 clients, so I cannot say that
I tested the case.
Compilation without "ppoll" gets a warning:
pgbench.c:5645:1: warning: ‘clear_socket’ defined but not used...
clear_socket(socket_array *sa, int fd, int idx)
The "clear_socket" function is not used in this case. I'd suggest to
remove it from the prototypes, remove or comment the unused implementation
in the select section, and keep the one in the ppoll section. Or use it if
it is needed. Or inline it in the "init_socket_array" function where it is
used just once.
"clear_socket" removed. Functionality inlined into "init_socket_array"
I'm not sure of the name "socket_array", because it is an array only in
the ppoll case. Maybe "sockets_t" or "socket_set"? Or something else?
Changed "socket_array" to "socket_set"
Maybe "init_socket_array" should be named "clear_...".
Renamed "init_socket_array" to "clear_socket_set"
I would suggest not to include sys/select.h when using ppoll, as it is a useless
include this case. I.e. move includes in the ifdef USE_PPOLL section?
Done. Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
Please do not remove comments, eg:
- /* identify which client sockets should be checked for input */
Restored.
On #endif or #else, especially large scope ones, I would have a comment to
say about what it is, eg at the minimum:
#else /* select(2) implementation */
#endif /* USE_PPOLL */
On:
+#if defined(USE_PPOLL)
+#ifdef POLLRDHUP
Use just one "ifdef" style?
There should be a comment about what this sections are providing, eg:
/* ppoll(2) implementation for "socket_array" functions */
There should be an empty line and possibly a comment explaining why
POLLRDHUP may not be there and/or why this define is necessary.
Removed unneeded #ifdef around POLLRDHUP
With select you always initialize the timeout, but not with ppoll.
Use a common approach in the implementations?
Fixed init & passing of timeout.
The "finishCon" function addition seems totally unrelated to this patch.
Although I agree that this function improves the code, it is refactoring
and does not really belong here.
Removed.
--
Fabien.
Thank you!!
doug
Attachments:
pgbench11-ppoll-v7.patch (application/octet-stream, 10.7K) download
diff --git a/configure.in b/configure.in
index 4d26034579..7b2b531675 100644
--- a/configure.in
+++ b/configure.in
@@ -1425,7 +1425,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 31ea6ca06e..05773d8bb1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -47,6 +47,9 @@
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -83,12 +86,28 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-/* max number of clients allowed */
+#if defined(HAVE_PPOLL)
+#define USE_PPOLL 1
+#endif
+
+#if defined(USE_PPOLL)
+/* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_array;
+/* unlimited number of clients */
+#define MAXCLIENTS -1
+#else
+/* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_array;
#ifdef FD_SETSIZE
+/* system limited max number of clients allowed */
#define MAXCLIENTS (FD_SETSIZE - 10)
#else
+/* max number of clients allowed */
#define MAXCLIENTS 1024
#endif
+#endif
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -502,6 +521,15 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static void finishCon(CState *st);
+static socket_array *alloc_socket_array(int count);
+static void clear_socket(socket_array *sa, int fd, int idx);
+static int error_on_socket(socket_array *sa, int idx, PGconn *con);
+static void free_socket_array(socket_array *sa);
+static int ignore_socket(socket_array *sa, int idx, PGconn *con);
+static void init_socket_array(socket_array *sa, int count);
+static void set_socket(socket_array *sa, int fd, int idx);
+static int wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -2704,6 +2732,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -2981,8 +3010,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (is_connect)
{
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
INSTR_TIME_SET_ZERO(now);
}
@@ -3019,11 +3047,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
case CSTATE_ABORTED:
case CSTATE_FINISHED:
- if (st->con != NULL)
- {
- PQfinish(st->con);
- st->con = NULL;
- }
+ finishCon(st);
return;
}
}
@@ -3171,13 +3195,7 @@ disconnect_all(CState *state, int length)
int i;
for (i = 0; i < length; i++)
- {
- if (state[i].con)
- {
- PQfinish(state[i].con);
- state[i].con = NULL;
- }
- }
+ finishCon(&state[i]);
}
/*
@@ -4507,7 +4525,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5189,6 +5207,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_array *sockets = alloc_socket_array(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5232,6 +5251,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5248,13 +5268,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
- /* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ init_socket_array(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5265,8 +5284,7 @@ threadRun(void *arg)
{
/* interrupt client that has not started a transaction */
st->state = CSTATE_FINISHED;
- PQfinish(st->con);
- st->con = NULL;
+ finishCon(st);
remains--;
}
else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
@@ -5304,7 +5322,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5341,7 +5359,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5352,11 +5370,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_array(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5365,7 +5379,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_array(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5376,7 +5390,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5385,7 +5399,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ init_socket_array(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5396,16 +5410,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5543,9 +5552,137 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_array(sockets);
+ sockets = NULL;
return NULL;
}
+static void
+finishCon(CState *st)
+{
+ if (st->con)
+ {
+ PQfinish(st->con);
+ st->con = NULL;
+ }
+}
+
+static socket_array *
+alloc_socket_array(int count)
+{
+ return (socket_array *) pg_malloc0(sizeof(socket_array) * ((MAXCLIENTS > 0) ? 1 : count));
+}
+
+static void
+free_socket_array(socket_array *sa)
+{
+ pg_free(sa);
+}
+
+#if defined(USE_PPOLL)
+#ifdef POLLRDHUP
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+#define POLL_ISSUES (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+#else
+#define POLL_EVENTS (POLLIN|POLLPRI)
+#define POLL_ISSUES (POLLERR|POLLHUP|POLLNVAL)
+#endif
+
+static void
+clear_socket(socket_array *sa, int fd, int idx)
+{
+ set_socket(sa, -1, idx);
+}
+
+static int
+error_on_socket(socket_array *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_ISSUES)))
+ return 0;
+ fprintf(stderr, "ppoll() fail - errno: %d, idx: %d, events: %x, %s\n",
+ errno, idx, (sa[idx].revents & POLL_ISSUES),
+ PQerrorMessage(con));
+ return 1;
+}
+
+static int
+ignore_socket(socket_array *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents) ? 1 : 0;
+}
+
+static void
+init_socket_array(socket_array *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ clear_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_array *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ }
+ return ppoll(sa, nstate, usec ? &timeout : NULL, NULL);
+}
+#else
+static void
+clear_socket(socket_array *sa, int fd, int idx)
+{
+ FD_CLR(fd, sa);
+}
+
+static int
+error_on_socket(socket_array *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return 0;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return 0;
+}
+
+static int
+ignore_socket(socket_array *sa, int idx, PGconn *con)
+{
+ return (!FD_ISSET(PQsocket(con), sa)) ? 1 : 0;
+}
+
+static void
+init_socket_array(socket_array *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_array *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_array(socket_array *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, usec ? &timeout : NULL);
+}
+#endif
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f98f773ff0..63207e13fd 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -409,6 +409,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:05 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-25 15:16 ` Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2018-01-25 15:16 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
ISTM that the v7 patch version you sent is identical to v6.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-01-25 15:34 ` Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
1 sibling, 1 reply; 51+ messages in thread
From: Rady, Doug @ 2018-01-25 15:34 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
This time with the revised patch file: pgbench11-ppoll-v8.patch
On 1/24/18, 00:00, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
Hello Doug,
Hello Fabien,
> This version of the patch attempts to address the feedback for the
> previous submission on 28-Nov-2017
Please avoid recreating a thread, but rather respond to the previous one,
I missed this post.
Got it.
The overall function-based implementation with limited ifdefs seems
readable and maintainable. I think that it rather improves the code in
places by hiding details.
Patch applies with one warning:
pgbench11-ppoll-v6.patch:141: trailing whitespace.
set_socket(sockets, PQsocket(state[i].con), i);<SPACE>
warning: 1 line adds whitespace errors.
Fixed.
The patch implies to run "autoconf" to regenerate the configure script.
Yes. I should have included this in the description.
Compilation with "ppoll" ok, globale & local make check ok. I do not have
hardware which allows to run with over 1024 clients, so I cannot say that
I tested the case.
Compilation without "ppoll" gets a warning:
pgbench.c:5645:1: warning: ‘clear_socket’ defined but not used...
clear_socket(socket_array *sa, int fd, int idx)
The "clear_socket" function is not used in this case. I'd suggest to
remove it from the prototypes, remove or comment the unused implementation
in the select section, and keep the one in the ppoll section. Or use it if
it is needed. Or inline it in the "init_socket_array" function where it is
used just once.
"clear_socket" removed. Functionality inlined into "init_socket_array"
I'm not sure of the name "socket_array", because it is an array only in
the ppoll case. Maybe "sockets_t" or "socket_set"? Or something else?
Changed "socket_array" to "socket_set"
Maybe "init_socket_array" should be named "clear_...".
Renamed "init_socket_array" to "clear_socket_set"
I would suggest not to include sys/select.h when using ppoll, as it is a useless
include this case. I.e. move includes in the ifdef USE_PPOLL section?
Done. Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
Please do not remove comments, eg:
- /* identify which client sockets should be checked for input */
Restored.
On #endif or #else, especially large scope ones, I would have a comment to
say about what it is, eg at the minimum:
#else /* select(2) implementation */
#endif /* USE_PPOLL */
On:
+#if defined(USE_PPOLL)
+#ifdef POLLRDHUP
Use just one "ifdef" style?
There should be a comment about what this sections are providing, eg:
/* ppoll(2) implementation for "socket_array" functions */
There should be an empty line and possibly a comment explaining why
POLLRDHUP may not be there and/or why this define is necessary.
Removed unneeded #ifdef around POLLRDHUP
With select you always initialize the timeout, but not with ppoll.
Use a common approach in the implementations?
Fixed init & passing of timeout.
The "finishCon" function addition seems totally unrelated to this patch.
Although I agree that this function improves the code, it is refactoring
and does not really belong here.
Removed.
--
Fabien.
Thank you!!
doug
Attachments:
pgbench11-ppoll-v8.patch (application/octet-stream, 9.6K) download
diff --git a/configure.in b/configure.in
index 4d26034579..7b2b531675 100644
--- a/configure.in
+++ b/configure.in
@@ -1425,7 +1425,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 31ea6ca06e..fb100abaaa 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -44,9 +44,13 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifdef HAVE_PPOLL
+#include <poll.h>
+#else
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -83,12 +87,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-/* max number of clients allowed */
+#ifdef HAVE_PPOLL /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#else /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#endif /* HAVE_PPOLL */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -502,6 +513,13 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static socket_set *alloc_socket_set(int count);
+static int error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static int ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1071,6 +1089,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -2704,6 +2723,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -4507,7 +4527,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5189,6 +5209,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5232,6 +5253,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5248,13 +5270,13 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5304,7 +5326,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5341,7 +5363,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5352,11 +5374,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5365,7 +5383,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5376,7 +5394,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5385,7 +5403,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5396,16 +5414,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5543,9 +5556,118 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * ((MAXCLIENTS > 0) ? 1 : count));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+#ifdef HAVE_PPOLL /* ppoll(2) based socket polling */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+#define POLL_ISSUES (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL|POLLRDHUP)
+
+static int
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_ISSUES)))
+ return 0;
+ fprintf(stderr, "ppoll() fail - errno: %d, idx: %d, events: %x, %s\n",
+ errno, idx, (sa[idx].revents & POLL_ISSUES),
+ PQerrorMessage(con));
+ return 1;
+}
+
+static int
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents) ? 1 : 0;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ } else {
+ timeout.tv_sec = timeout.tv_nsec = 0;
+ }
+ return ppoll(sa, nstate, &timeout, NULL);
+}
+#else /* select(2) based socket polling */
+static int
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return 0;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return 0;
+}
+
+static int
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (!FD_ISSET(PQsocket(con), sa)) ? 1 : 0;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ } else {
+ timeout.tv_sec = timeout.tv_usec = 0;
+ }
+
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+}
+#endif /* HAVE_PPOLL */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f98f773ff0..63207e13fd 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -409,6 +409,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-25 22:46 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2018-01-25 22:46 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
> This time with the revised patch file: pgbench11-ppoll-v8.patch
Patch applies cleanly. Compiles cleanly and runs fine in both ppoll &
select cases.
I'm okay with having a preferred ppoll implementation because of its improved
capability.
A few minor additional comments/suggestions:
Cpp has an #elif that could be used to manage the ppoll/select alternative.
It is already used elsewhere in the file. Or not.
I must admit that I'm not fond of the alloc_socket_set trick with MAXCLIENTS,
especially without any comment. I'd suggest to just have two distinct functions
in their corresponding sections.
I would add a comment that free_socket_set code is common to both
versions, and maybe consider moving it afterwards. Or maybe just duplicate
if in each section for homogeneity.
It looks like error_on_socket and ignore_socket should return a boolean instead
of an int. Also, maybe simplify the implementation of the later by avoiding
the ?: expression.
ISTM that the error_on_socket function in the ppoll case deserves some
comments, especially on the condition.
> [...] Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
I'm okay with that. I'm wondering whether there should be a way to force
using one or the other when both are available. Not sure.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-01-26 22:27 ` Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Rady, Doug @ 2018-01-26 22:27 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 1/25/18, 14:46, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
Hello Doug,
Hello Fabien,
> This time with the revised patch file: pgbench11-ppoll-v8.patch
Patch applies cleanly. Compiles cleanly and runs fine in both ppoll &
select cases.
I'm okay with having a preferred ppoll implementation because of its improved
capability.
A few minor additional comments/suggestions:
Cpp has an #elif that could be used to manage the ppoll/select alternative.
It is already used elsewhere in the file. Or not.
I must admit that I'm not fond of the alloc_socket_set trick with MAXCLIENTS,
especially without any comment. I'd suggest to just have two distinct functions
in their corresponding sections.
Made a distinct function for each section.
I would add a comment that free_socket_set code is common to both
versions, and maybe consider moving it afterwards. Or maybe just duplicate
if in each section for homogeneity.
Duplicated.
It looks like error_on_socket and ignore_socket should return a boolean instead
of an int. Also, maybe simplify the implementation of the later by avoiding
the ?: expression.
ISTM that the error_on_socket function in the ppoll case deserves some
comments, especially on the condition.
Added comment. Changed output to be more compatible with socket() error check.
Extra ppoll() specific error info only output when debug flag set ... not sure if this is OK or should just remove the extra info.
> [...] Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
I'm okay with that. I'm wondering whether there should be a way to force
using one or the other when both are available. Not sure.
Added option to force use of select(2) via: -DUSE_SELECT
--
Fabien.
Thank you!!
doug
Attachments:
pgbench11-ppoll-v9.patch (application/octet-stream, 10.3K) download
diff --git a/configure.in b/configure.in
index 4d26034579..7b2b531675 100644
--- a/configure.in
+++ b/configure.in
@@ -1425,7 +1425,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 31ea6ca06e..2ec8d09561 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -44,7 +44,14 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
-#ifdef HAVE_SYS_SELECT_H
+#ifdef USE_SELECT /* force use of select(2)? */
+#undef HAVE_PPOLL
+#endif
+#ifdef HAVE_PPOLL
+#include <poll.h>
+#elif defined(HAVE_SYS_SELECT_H)
+#undef USE_SELECT
+#define USE_SELECT
#include <sys/select.h>
#endif
@@ -82,13 +89,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef USE_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* USE_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -502,6 +515,13 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1071,6 +1091,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -2704,6 +2725,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -4507,7 +4529,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5189,6 +5211,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5232,6 +5255,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5248,13 +5272,13 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5304,7 +5328,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5341,7 +5365,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5352,11 +5376,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5365,7 +5385,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5376,7 +5396,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5385,7 +5405,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5396,16 +5416,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5543,9 +5558,143 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
+#ifdef USE_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * MAXCLIENTS);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (FD_ISSET(PQsocket(con), sa))
+ return false;
+ else
+ return true;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ } else {
+ timeout.tv_sec = timeout.tv_usec = 0;
+ }
+
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd != -1 && !sa[idx].revents)
+ return true;
+ else
+ return false;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ } else {
+ timeout.tv_sec = timeout.tv_nsec = 0;
+ }
+ return ppoll(sa, nstate, &timeout, NULL);
+}
+#endif /* USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f98f773ff0..63207e13fd 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -409,6 +409,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-26 23:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2018-01-26 23:00 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
Patch applies, compiles, tests ok.
> > [...] Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
>
> I'm okay with that. I'm wondering whether there should be a way to force
> using one or the other when both are available. Not sure.
>
> Added option to force use of select(2) via: -DUSE_SELECT
USE_SELECT could mean something somewhere. Maybe use something more
specific like PGBENCH_USE_SELECT? Having this macro available simplifies
testing.
I'm not sure why you do the following trick, could you explain?
+#undef USE_SELECT
+#define USE_SELECT
In the select implementation you do:
return (socket_set *) pg_malloc0(sizeof(socket_set) * MAXCLIENTS);
but ISTM that socket_set is already an fd_set which represents a set of
clients, so allocating a number of it is needed. The initial
implementation just does "fs_set input_mask", whetever the number of
clients, and it works fine.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-01-28 23:02 ` Rady, Doug <radydoug@amazon.com>
2018-01-29 08:58 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 51+ messages in thread
From: Rady, Doug @ 2018-01-28 23:02 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 1/26/18, 15:00, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
Hello Doug,
Hello Fabien,
Patch applies, compiles, tests ok.
> > [...] Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant.
>
> I'm okay with that. I'm wondering whether there should be a way to force
> using one or the other when both are available. Not sure.
>
> Added option to force use of select(2) via: -DUSE_SELECT
USE_SELECT could mean something somewhere. Maybe use something more
specific like PGBENCH_USE_SELECT? Having this macro available simplifies
testing.
Changed to PGBENCH_USE_SELECT
I'm not sure why you do the following trick, could you explain?
+#undef USE_SELECT
+#define USE_SELECT
This was due to compiler complaint about USE_SELECT being redefined.
Have replaced that "trick" with a new #define POLL_USING_SELECT which is used elsewhere in pgbench instead.
In the select implementation you do:
return (socket_set *) pg_malloc0(sizeof(socket_set) * MAXCLIENTS);
but ISTM that socket_set is already an fd_set which represents a set of
clients, so allocating a number of it is needed. The initial
implementation just does "fs_set input_mask", whetever the number of
clients, and it works fine.
Ugh. Yes, for socket() only one (1) fd_set is needed.
Fixed.
--
Fabien.
Thank you, again!!
doug
Attachments:
pgbench11-ppoll-v10.patch (application/octet-stream, 10.3K) download
diff --git a/configure.in b/configure.in
index 4d26034579..7b2b531675 100644
--- a/configure.in
+++ b/configure.in
@@ -1425,7 +1425,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 31ea6ca06e..689f15a772 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -44,7 +44,13 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
-#ifdef HAVE_SYS_SELECT_H
+#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */
+#undef HAVE_PPOLL
+#endif
+#ifdef HAVE_PPOLL
+#include <poll.h>
+#elif defined(HAVE_SYS_SELECT_H)
+#define POLL_USING_SELECT
#include <sys/select.h>
#endif
@@ -82,13 +88,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -502,6 +514,13 @@ static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1071,6 +1090,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -2704,6 +2724,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -4507,7 +4528,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5189,6 +5210,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5232,6 +5254,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5248,13 +5271,13 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5304,7 +5327,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5341,7 +5364,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5352,11 +5375,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5365,7 +5384,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5376,7 +5395,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5385,7 +5404,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5396,16 +5415,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5543,9 +5557,143 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (FD_ISSET(PQsocket(con), sa))
+ return false;
+ else
+ return true;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ } else {
+ timeout.tv_sec = timeout.tv_usec = 0;
+ }
+
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd != -1 && !sa[idx].revents)
+ return true;
+ else
+ return false;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ } else {
+ timeout.tv_sec = timeout.tv_nsec = 0;
+ }
+ return ppoll(sa, nstate, &timeout, NULL);
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f98f773ff0..63207e13fd 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -409,6 +409,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-01-29 08:58 ` Fabien COELHO <coelho@cri.ensmp.fr>
1 sibling, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2018-01-29 08:58 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
> I'm not sure why you do the following trick, could you explain?
> +#undef USE_SELECT
> +#define USE_SELECT
>
> This was due to compiler complaint about USE_SELECT being redefined.
> Have replaced that "trick" with a new #define POLL_USING_SELECT which is used elsewhere in pgbench instead.
Ok, why not.
Another option to avoid the warning and a new name could have been to
"#ifndef X #define X #endif /* !X */"
Patch applies cleanly, compiles cleanly for both options, local & global
"make check" ok.
Switched to "Ready".
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-03-01 08:34 ` Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
1 sibling, 1 reply; 51+ messages in thread
From: Andres Freund @ 2018-03-01 08:34 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 2018-01-28 23:02:57 +0000, Rady, Doug wrote:
> diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
> index 31ea6ca06e..689f15a772 100644
> --- a/src/bin/pgbench/pgbench.c
> +++ b/src/bin/pgbench/pgbench.c
> @@ -44,7 +44,13 @@
> #include <signal.h>
> #include <time.h>
> #include <sys/time.h>
> -#ifdef HAVE_SYS_SELECT_H
> +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */
> +#undef HAVE_PPOLL
> +#endif
> +#ifdef HAVE_PPOLL
> +#include <poll.h>
> +#elif defined(HAVE_SYS_SELECT_H)
> +#define POLL_USING_SELECT
(random thing noticed while going through patches)
It strikes me as a bad idea to undefine configure selected
symbols. Postgres header might rely on them. It also strikes me as
entirely unnecessary here.
- Andres
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
@ 2018-03-01 10:30 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2018-03-01 10:30 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
>> -#ifdef HAVE_SYS_SELECT_H
>> +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */
>> +#undef HAVE_PPOLL
>> +#endif
>> +#ifdef HAVE_PPOLL
>> +#include <poll.h>
>> +#elif defined(HAVE_SYS_SELECT_H)
>> +#define POLL_USING_SELECT
>
> (random thing noticed while going through patches)
>
> It strikes me as a bad idea to undefine configure selected
> symbols. Postgres header might rely on them. It also strikes me as
> entirely unnecessary here.
Yes, I though about this one but let it pass. Indeed, it would be
sufficient to not load "poll.h" when select is forced, without undefining
the configure setting.
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-03-04 00:14 ` Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
0 siblings, 1 reply; 51+ messages in thread
From: Andres Freund @ 2018-03-04 00:14 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Rady, Doug <radydoug@amazon.com>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 2018-03-01 11:30:39 +0100, Fabien COELHO wrote:
>
> > > -#ifdef HAVE_SYS_SELECT_H
> > > +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */
> > > +#undef HAVE_PPOLL
> > > +#endif
> > > +#ifdef HAVE_PPOLL
> > > +#include <poll.h>
> > > +#elif defined(HAVE_SYS_SELECT_H)
> > > +#define POLL_USING_SELECT
> >
> > (random thing noticed while going through patches)
> >
> > It strikes me as a bad idea to undefine configure selected
> > symbols. Postgres header might rely on them. It also strikes me as
> > entirely unnecessary here.
>
> Yes, I though about this one but let it pass. Indeed, it would be sufficient
> to not load "poll.h" when select is forced, without undefining the configure
> setting.
I've marked the CF entry waiting on author.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
@ 2018-03-21 05:42 ` Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 1 reply; 51+ messages in thread
From: Rady, Doug @ 2018-03-21 05:42 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Updated the patch to not do the #undef
pgbench11-ppoll-v11.patch attached.
Thanks,
doug
On 3/3/18, 16:14, "Andres Freund" <andres@anarazel.de> wrote:
On 2018-03-01 11:30:39 +0100, Fabien COELHO wrote:
>
> > > -#ifdef HAVE_SYS_SELECT_H
> > > +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */
> > > +#undef HAVE_PPOLL
> > > +#endif
> > > +#ifdef HAVE_PPOLL
> > > +#include <poll.h>
> > > +#elif defined(HAVE_SYS_SELECT_H)
> > > +#define POLL_USING_SELECT
> >
> > (random thing noticed while going through patches)
> >
> > It strikes me as a bad idea to undefine configure selected
> > symbols. Postgres header might rely on them. It also strikes me as
> > entirely unnecessary here.
>
> Yes, I though about this one but let it pass. Indeed, it would be sufficient
> to not load "poll.h" when select is forced, without undefining the configure
> setting.
I've marked the CF entry waiting on author.
Greetings,
Andres Freund
Attachments:
pgbench11-ppoll-v11.patch (application/octet-stream, 10.3K) download
--- a/configure.in
+++ b/configure.in
@@ -1520,7 +1520,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 29d69de4d1..45177af5ac 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -44,9 +44,18 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+#ifdef HAVE_PPOLL
+#define POLL_USING_PPOLL
+#include <poll.h>
+#endif
+#endif
+#ifndef POLL_USING_PPOLL
#ifdef HAVE_SYS_SELECT_H
+#define POLL_USING_SELECT
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -82,13 +91,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -503,6 +518,13 @@ static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1072,6 +1094,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -2706,6 +2729,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (INSTR_TIME_IS_ZERO(now))
INSTR_TIME_SET_CURRENT(now);
start = now;
+
if ((st->con = doConnect()) == NULL)
{
fprintf(stderr, "client %d aborted while establishing connection\n",
@@ -4498,7 +4522,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5180,6 +5204,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5223,6 +5248,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5239,13 +5265,13 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
+
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5294,7 +5320,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5331,7 +5357,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5342,11 +5368,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5355,7 +5377,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5366,7 +5388,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5375,7 +5397,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5386,16 +5408,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5533,6 +5550,8 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
@@ -5546,6 +5565,138 @@ finishCon(CState *st)
}
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (FD_ISSET(PQsocket(con), sa))
+ return false;
+ else
+ return true;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ } else {
+ timeout.tv_sec = timeout.tv_usec = 0;
+ }
+
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (sa[idx].fd != -1 && !sa[idx].revents)
+ return true;
+ else
+ return false;
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ } else {
+ timeout.tv_sec = timeout.tv_nsec = 0;
+ }
+ return ppoll(sa, nstate, &timeout, NULL);
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index c5db5430b9..a12a08892d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -425,6 +425,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-03-25 11:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
0 siblings, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2018-03-25 11:00 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
> Updated the patch to not do the #undef
> pgbench11-ppoll-v11.patch attached.
Patch applies. Do not forget to regenerate configure to test...
I've compiled and run with both ppoll & select options without issues.
Two quite minor style comment (at least 2 instances):
if (cond) return false; else return true;
ISTM that the simpler the better:
return !cond;
Also ISTM that the following does not comply with pg C style expectations
(idem, 2 instances):
} else {
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-03-26 20:23 ` Rady, Doug <radydoug@amazon.com>
2018-03-27 09:06 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-04-06 21:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
0 siblings, 3 replies; 51+ messages in thread
From: Rady, Doug @ 2018-03-26 20:23 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 3/25/18, 04:00, "Fabien COELHO" <coelho@cri.ensmp.fr> wrote:
Hello Fabien,
Hello Doug,
> Updated the patch to not do the #undef
> pgbench11-ppoll-v11.patch attached.
Patch applies. Do not forget to regenerate configure to test...
I've compiled and run with both ppoll & select options without issues.
Two quite minor style comment (at least 2 instances):
if (cond) return false; else return true;
ISTM that the simpler the better:
return !cond;
Fixed.
Also ISTM that the following does not comply with pg C style expectations
(idem, 2 instances):
} else {
Fixed.
Also fixed issue with 'timeout' not being passed as NULL when no timeout time.
--
Fabien.
Thanks!
doug
Attachments:
pgbench11-ppoll-v12.patch (application/octet-stream, 10.0K) download
--- a/configure.in
+++ b/configure.in
@@ -1528,7 +1528,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 8529e7dc47..00a865769f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -45,9 +45,18 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+#ifdef HAVE_PPOLL
+#define POLL_USING_PPOLL
+#include <poll.h>
+#endif
+#endif
+#ifndef POLL_USING_PPOLL
#ifdef HAVE_SYS_SELECT_H
+#define POLL_USING_SELECT
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -91,13 +100,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -524,6 +539,13 @@ static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1142,6 +1164,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -4860,7 +4883,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5567,6 +5590,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5610,6 +5634,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5626,13 +5651,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5681,7 +5705,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5718,7 +5742,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5729,11 +5753,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5742,7 +5762,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5753,7 +5773,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5762,7 +5782,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5773,16 +5793,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5920,6 +5935,8 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
@@ -5933,6 +5950,135 @@ finishCon(CState *st)
}
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return !(FD_ISSET(PQsocket(con), sa));
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(maxsock + 1, sa, NULL, NULL, NULL);
+ }
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents);
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ return ppoll(sa, nstate, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa, nstate, NULL, NULL);
+ }
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 625cd21ed3..39adca53ab 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -425,6 +425,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-03-27 09:06 ` Fabien COELHO <coelho@cri.ensmp.fr>
2 siblings, 0 replies; 51+ messages in thread
From: Fabien COELHO @ 2018-03-27 09:06 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hello Doug,
>> I've compiled and run with both ppoll & select options without issues.
>> Two quite minor style comment (at least 2 instances):
> Fixed. Fixed. Also fixed issue with 'timeout' not being passed as NULL
> when no timeout time.
v12 compiled and tested with both ppoll & select (by forcing). All seems
ok to me.
Switched back to "ready".
--
Fabien.
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-04-06 21:15 ` Andres Freund <andres@anarazel.de>
2018-04-06 21:49 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 51+ messages in thread
From: Andres Freund @ 2018-04-06 21:15 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Hi,
I'm still not particularly happy with this. Checking whether I can
polish it up.
a) the new function names are partially non-descriptive and their
meaning is undocumented. As an extreme example:
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
reading the new code it's entirely unclear what that could mean. Are
you marking the socket as ignored? What does ignored even mean?
There's not a single comment on what the new functions mean. It's not
that bad if there's no docs on what FD_ISSET implies, because that's a
well known and documented interface. But introducing an abstraction
without any comments on it?
b) Does this actually improve the situation all that much? We still loop
over every socket:
/* ok, advance the state machine of each connection */
for (i = 0; i < nstate; i++)
{
CState *st = &state[i];
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
if (error_on_socket(sockets, i, st->con))
goto done;
if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
st->state == CSTATE_ABORTED)
{
/* this client is done, no need to consider it anymore */
continue;
}
doCustom(thread, st, &aggs);
/* If doCustom changed client to finished state, reduce remains */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
remains--;
}
if the goal is to make this more scalable, wouldn't this require
using a proper polling mechanism that supports signalling the
sockets with relevant changes, rather than busy-looping through every
socket if there's just a single change?
I kinda wonder if the proper fix wouldn't be to have one patch making
WaitEventSets usable from frontend code, and then make this code use
them. Not a small project though.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-04-06 21:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
@ 2018-04-06 21:49 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-04-06 21:54 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-04-07 05:20 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts konstantin knizhnik <k.knizhnik@postgrespro.ru>
0 siblings, 2 replies; 51+ messages in thread
From: Tom Lane @ 2018-04-06 21:49 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Rady, Doug <radydoug@amazon.com>; Fabien COELHO <coelho@cri.ensmp.fr>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Andres Freund <andres@anarazel.de> writes:
> I'm still not particularly happy with this.
I'm a bit confused as to what the point is. It seems unlikely that one
pgbench process can effectively drive enough backends for select's
limitations to really be an issue.
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-04-06 21:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-04-06 21:49 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
@ 2018-04-06 21:54 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 51+ messages in thread
From: Andres Freund @ 2018-04-06 21:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Rady, Doug <radydoug@amazon.com>; Fabien COELHO <coelho@cri.ensmp.fr>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 2018-04-06 17:49:19 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I'm still not particularly happy with this.
>
> I'm a bit confused as to what the point is. It seems unlikely that one
> pgbench process can effectively drive enough backends for select's
> limitations to really be an issue.
It's not that crazy to use more than 1024 connections (common FD_SETSIZE
valu), especially over a higher latency connection.
As I wrote, I think using a poll API that doesn't require looping over
all sockets, even if they didn't get an event, would be a better plan.
- Andres
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-04-06 21:15 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-04-06 21:49 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
@ 2018-04-07 05:20 ` konstantin knizhnik <k.knizhnik@postgrespro.ru>
1 sibling, 0 replies; 51+ messages in thread
From: konstantin knizhnik @ 2018-04-07 05:20 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Rady, Doug <radydoug@amazon.com>; Fabien COELHO <coelho@cri.ensmp.fr>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On Apr 7, 2018, at 12:49 AM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> I'm still not particularly happy with this.
>
> I'm a bit confused as to what the point is. It seems unlikely that one
> pgbench process can effectively drive enough backends for select's
> limitations to really be an issue.
pgbench is multithreaded application, so in theory it can drive almost arbitrary number of connections.
It is limited only by network throughput, but if pgbench is launched at the same host and connected to the server through loopback or unix sockets,
then network is also not a limit.
We quite often have to spawn more than 1k connections and SMP systems with hundreds of CPU.
So there are two choices: either use patched version of pgbench which is using poll, either spawn several instances of pgbench (which is not always convenient).
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-05-17 05:23 ` Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2 siblings, 1 reply; 51+ messages in thread
From: Thomas Munro @ 2018-05-17 05:23 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On Tue, Mar 27, 2018 at 9:23 AM, Rady, Doug <radydoug@amazon.com> wrote:
> pgbench11-ppoll-v12.patch
Hi Doug,
FYI this patch is trying and failing to use ppoll() on Windows:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.30
--
Thomas Munro
http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
@ 2018-07-03 23:52 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
0 siblings, 1 reply; 51+ messages in thread
From: Andrew Dunstan @ 2018-07-03 23:52 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 05/17/2018 01:23 AM, Thomas Munro wrote:
> On Tue, Mar 27, 2018 at 9:23 AM, Rady, Doug <radydoug@amazon.com> wrote:
>> pgbench11-ppoll-v12.patch
> Hi Doug,
>
> FYI this patch is trying and failing to use ppoll() on Windows:
>
> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.30
>
It's still failing - see
<https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.4098;
I'm setting this back to "Waiting on Author" until that's fixed.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
@ 2018-08-09 16:45 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
0 siblings, 1 reply; 51+ messages in thread
From: Andrew Dunstan @ 2018-08-09 16:45 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 07/03/2018 07:52 PM, Andrew Dunstan wrote:
>
>
> On 05/17/2018 01:23 AM, Thomas Munro wrote:
>> On Tue, Mar 27, 2018 at 9:23 AM, Rady, Doug <radydoug@amazon.com> wrote:
>>> pgbench11-ppoll-v12.patch
>> Hi Doug,
>>
>> FYI this patch is trying and failing to use ppoll() on Windows:
>>
>> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.30
>>
>
>
> It's still failing - see
> <https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.4098;
>
> I'm setting this back to "Waiting on Author" until that's fixed.
>
The author hasn't replied, but the attached seems to have cured the
bitrot so that it at least applies. Let's see what the cfbot makes of it
and then possibly fix any Windows issues.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
pgbench-ppoll-v13.patch (text/x-patch, 10.9K) download
diff --git a/configure b/configure
index 2665213..4579003 100755
--- a/configure
+++ b/configure
@@ -14916,7 +14916,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat ppoll pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 397f6bc..bd08068 100644
--- a/configure.in
+++ b/configure.in
@@ -1540,7 +1540,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat ppoll pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 41b756c..e23ba1a 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -45,9 +45,18 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+#ifdef HAVE_PPOLL
+#define POLL_USING_PPOLL
+#include <poll.h>
+#endif
+#endif
+#ifndef POLL_USING_PPOLL
#ifdef HAVE_SYS_SELECT_H
+#define POLL_USING_SELECT
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -92,13 +101,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -525,6 +540,13 @@ static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1143,6 +1165,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -4903,7 +4926,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5614,6 +5637,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5657,6 +5681,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5673,13 +5698,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5728,7 +5752,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5765,7 +5789,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5776,11 +5800,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5789,7 +5809,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5800,7 +5820,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5809,7 +5829,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5820,16 +5840,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5967,6 +5982,8 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
@@ -5980,6 +5997,135 @@ finishCon(CState *st)
}
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return !(FD_ISSET(PQsocket(con), sa));
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(maxsock + 1, sa, NULL, NULL, NULL);
+ }
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents);
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ return ppoll(sa, nstate, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa, nstate, NULL, NULL);
+ }
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index b7e4696..5e0e5f3 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -434,6 +434,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
@ 2018-08-09 21:46 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-10 01:32 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-15 13:59 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
0 siblings, 2 replies; 51+ messages in thread
From: Andrew Dunstan @ 2018-08-09 21:46 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 08/09/2018 12:45 PM, Andrew Dunstan wrote:
>
>
> On 07/03/2018 07:52 PM, Andrew Dunstan wrote:
>>
>>
>> On 05/17/2018 01:23 AM, Thomas Munro wrote:
>>> On Tue, Mar 27, 2018 at 9:23 AM, Rady, Doug <radydoug@amazon.com>
>>> wrote:
>>>> pgbench11-ppoll-v12.patch
>>> Hi Doug,
>>>
>>> FYI this patch is trying and failing to use ppoll() on Windows:
>>>
>>> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.30
>>>
>>>
>>
>>
>> It's still failing - see
>> <https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.4098;
>>
>> I'm setting this back to "Waiting on Author" until that's fixed.
>>
>
>
> The author hasn't replied, but the attached seems to have cured the
> bitrot so that it at least applies. Let's see what the cfbot makes of
> it and then possibly fix any Windows issues.
>
>
>
And there's still a Windows problem which I think is cured in the
attached patch
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
pgbench-ppoll-v14.patch (text/x-patch, 10.9K) download
diff --git a/configure b/configure
index 2665213..4579003 100755
--- a/configure
+++ b/configure
@@ -14916,7 +14916,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat ppoll pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 397f6bc..bd08068 100644
--- a/configure.in
+++ b/configure.in
@@ -1540,7 +1540,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat ppoll pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 41b756c..3d378db 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -45,9 +45,18 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+#ifdef HAVE_PPOLL
+#define POLL_USING_PPOLL
+#include <poll.h>
+#endif
+#endif
+#ifndef POLL_USING_PPOLL
+#define POLL_USING_SELECT
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -92,13 +101,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -525,6 +540,13 @@ static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1143,6 +1165,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -4903,7 +4926,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5614,6 +5637,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5657,6 +5681,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5673,13 +5698,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5728,7 +5752,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5765,7 +5789,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5776,11 +5800,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5789,7 +5809,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5800,7 +5820,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5809,7 +5829,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5820,16 +5840,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5967,6 +5982,8 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
@@ -5980,6 +5997,135 @@ finishCon(CState *st)
}
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return !(FD_ISSET(PQsocket(con), sa));
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(maxsock + 1, sa, NULL, NULL, NULL);
+ }
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents);
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ return ppoll(sa, nstate, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa, nstate, NULL, NULL);
+ }
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index b7e4696..5e0e5f3 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -434,6 +434,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
@ 2018-08-10 01:32 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
1 sibling, 0 replies; 51+ messages in thread
From: Andrew Dunstan @ 2018-08-10 01:32 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; +Cc: Fabien COELHO <coelho@cri.ensmp.fr>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
On 08/09/2018 05:46 PM, Andrew Dunstan wrote:
>
>
> On 08/09/2018 12:45 PM, Andrew Dunstan wrote:
>>
>>
>> On 07/03/2018 07:52 PM, Andrew Dunstan wrote:
>>>
>>>
>>> On 05/17/2018 01:23 AM, Thomas Munro wrote:
>>>> On Tue, Mar 27, 2018 at 9:23 AM, Rady, Doug <radydoug@amazon.com>
>>>> wrote:
>>>>> pgbench11-ppoll-v12.patch
>>>> Hi Doug,
>>>>
>>>> FYI this patch is trying and failing to use ppoll() on Windows:
>>>>
>>>> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.30
>>>>
>>>>
>>>
>>>
>>> It's still failing - see
>>> <https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.4098;
>>>
>>> I'm setting this back to "Waiting on Author" until that's fixed.
>>>
>>
>>
>> The author hasn't replied, but the attached seems to have cured the
>> bitrot so that it at least applies. Let's see what the cfbot makes of
>> it and then possibly fix any Windows issues.
>>
>>
>>
>
>
> And there's still a Windows problem which I think is cured in the
> attached patch
>
and the CFBot agrees.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
@ 2018-09-15 13:59 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-09-22 16:09 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 51+ messages in thread
From: Fabien COELHO @ 2018-09-15 13:59 UTC (permalink / raw)
To: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>; +Cc: Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
>> The author hasn't replied, but the attached seems to have cured the
>> bitrot so that it at least applies. Let's see what the cfbot makes of
>> it and then possibly fix any Windows issues.
The patch was not applying cleanly anymore for me, so here is a rebase of
your latest version.
Morever, ISTM that Tom's "why?" question has been answered: there are very
large systems out there with many processors, which are tested against
many connections, exceeding select limit.
I have turned back this patch to ready.
--
Fabien.
Attachments:
pgbench-ppoll-15.patch (text/x-diff, 11.3K) download
diff --git a/configure b/configure
index c6a44a9078..0456064456 100755
--- a/configure
+++ b/configure
@@ -15060,7 +15060,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 3ada48b5f9..fceba79023 100644
--- a/configure.in
+++ b/configure.in
@@ -1544,7 +1544,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 41b756c089..3d378db714 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -45,9 +45,18 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+#ifdef HAVE_PPOLL
+#define POLL_USING_PPOLL
+#include <poll.h>
+#endif
+#endif
+#ifndef POLL_USING_PPOLL
+#define POLL_USING_SELECT
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+#endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
@@ -92,13 +101,19 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-
-/* max number of clients allowed */
+#ifdef POLL_USING_SELECT /* using select(2) */
+#define SOCKET_WAIT_METHOD "select"
+typedef fd_set socket_set;
#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
+#define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
-#define MAXCLIENTS 1024
+#define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+#else /* using ppoll(2) */
+#define SOCKET_WAIT_METHOD "ppoll"
+typedef struct pollfd socket_set;
+#define MAXCLIENTS -1 /* unlimited number of clients */
+#endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
@@ -525,6 +540,13 @@ static void addScript(ParsedScript script);
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+static socket_set *alloc_socket_set(int count);
+static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+static void free_socket_set(socket_set *sa);
+static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+static void clear_socket_set(socket_set *sa, int count);
+static void set_socket(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
@@ -1143,6 +1165,7 @@ doConnect(void)
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
@@ -4903,7 +4926,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5614,6 +5637,7 @@ threadRun(void *arg)
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
@@ -5657,6 +5681,7 @@ threadRun(void *arg)
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
@@ -5673,13 +5698,12 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
@@ -5728,7 +5752,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
+ set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
@@ -5765,7 +5789,7 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
+ * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
@@ -5776,11 +5800,7 @@ threadRun(void *arg)
{
if (maxsock != -1)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
@@ -5789,7 +5809,7 @@ threadRun(void *arg)
}
else /* no explicit delay, select without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
@@ -5800,7 +5820,7 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
@@ -5809,7 +5829,7 @@ threadRun(void *arg)
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
@@ -5820,16 +5840,11 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
- if (sock < 0)
- {
- fprintf(stderr, "invalid socket: %s",
- PQerrorMessage(st->con));
+ if (error_on_socket(sockets, i, st->con))
goto done;
- }
- if (!FD_ISSET(sock, &input_mask))
+ if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5967,6 +5982,8 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
@@ -5980,6 +5997,135 @@ finishCon(CState *st)
}
}
+#ifdef POLL_USING_SELECT /* select(2) based socket polling */
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return !(FD_ISSET(PQsocket(con), sa));
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ FD_ZERO(sa);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ FD_SET(fd, sa);
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(maxsock + 1, sa, NULL, NULL, NULL);
+ }
+}
+#else /* ppoll(2) based socket polling */
+/* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+/* ppoll() events returned that we do not want/expect to see. */
+#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static bool
+error_on_socket(socket_set *sa, int idx, PGconn *con)
+{
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+}
+
+static bool
+ignore_socket(socket_set *sa, int idx, PGconn *con)
+{
+ return (sa[idx].fd != -1 && !sa[idx].revents);
+}
+
+static void
+clear_socket_set(socket_set *sa, int count)
+{
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+}
+
+static void
+set_socket(socket_set *sa, int fd, int idx)
+{
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+{
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ return ppoll(sa, nstate, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa, nstate, NULL, NULL);
+ }
+}
+#endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 4094e22776..5d4079609c 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -443,6 +443,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-15 13:59 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
@ 2018-09-22 16:09 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-23 00:11 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 51+ messages in thread
From: Tom Lane @ 2018-09-22 16:09 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>; Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
Fabien COELHO <coelho@cri.ensmp.fr> writes:
> The patch was not applying cleanly anymore for me, so here is a rebase of
> your latest version.
The cfbot doesn't like that patch, probably because of the Windows newlines.
Here's a version with regular newlines, and some cosmetic cleanup in the
configure infrastructure.
I haven't looked at the pgbench changes proper yet, but I did quickly
test building on FreeBSD 11, which has ppoll, and it falls over:
pgbench.c:6080:69: error: use of undeclared identifier 'POLLRDHUP'
...== -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
^
pgbench.c:6059:24: note: expanded from macro 'POLL_UNWANTED'
#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
^
pgbench.c:6085:42: error: use of undeclared identifier 'POLLRDHUP'
errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
^
pgbench.c:6059:24: note: expanded from macro 'POLL_UNWANTED'
#define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
^
pgbench.c:6107:19: error: use of undeclared identifier 'POLLRDHUP'
sa[idx].events = POLL_EVENTS;
^
pgbench.c:6057:22: note: expanded from macro 'POLL_EVENTS'
#define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
^
3 errors generated.
make[3]: *** [<builtin>: pgbench.o] Error 1
I'm strongly tempted to just remove the POLL_UNWANTED business
altogether, as it seems both pointless and unportable on its face.
Almost by definition, we can't know what "other" bits a given
implementation might set.
I'm not entirely following the point of including POLLRDHUP in
POLL_EVENTS, either. What's wrong with the traditional solution
of detecting EOF?
regards, tom lane
Attachments:
pgbench-ppoll-16.patch (text/x-diff, 14.8K) download
diff --git a/configure b/configure
index 9b30402..21ecd29 100755
*** a/configure
--- b/configure
*************** fi
*** 15093,15099 ****
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
! for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
--- 15093,15099 ----
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
! for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 2e60a89..8fe6894 100644
*** a/configure.in
--- b/configure.in
*************** PGAC_FUNC_WCSTOMBS_L
*** 1562,1568 ****
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
! AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
--- 1562,1568 ----
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
! AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 41b756c..3d378db 100644
*** a/src/bin/pgbench/pgbench.c
--- b/src/bin/pgbench/pgbench.c
***************
*** 45,53 ****
--- 45,62 ----
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+ #ifndef PGBENCH_USE_SELECT /* force use of select(2)? */
+ #ifdef HAVE_PPOLL
+ #define POLL_USING_PPOLL
+ #include <poll.h>
+ #endif
+ #endif
+ #ifndef POLL_USING_PPOLL
+ #define POLL_USING_SELECT
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
+ #endif
#ifdef HAVE_SYS_RESOURCE_H
#include <sys/resource.h> /* for getrlimit */
*************** static int pthread_join(pthread_t th, vo
*** 92,104 ****
/********************************************************************
* some configurable parameters */
!
! /* max number of clients allowed */
#ifdef FD_SETSIZE
! #define MAXCLIENTS (FD_SETSIZE - 10)
#else
! #define MAXCLIENTS 1024
#endif
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
--- 101,119 ----
/********************************************************************
* some configurable parameters */
! #ifdef POLL_USING_SELECT /* using select(2) */
! #define SOCKET_WAIT_METHOD "select"
! typedef fd_set socket_set;
#ifdef FD_SETSIZE
! #define MAXCLIENTS (FD_SETSIZE - 10) /* system limited max number of clients allowed */
#else
! #define MAXCLIENTS 1024 /* max number of clients allowed */
#endif
+ #else /* using ppoll(2) */
+ #define SOCKET_WAIT_METHOD "ppoll"
+ typedef struct pollfd socket_set;
+ #define MAXCLIENTS -1 /* unlimited number of clients */
+ #endif /* POLL_USING_SELECT */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
*************** static void addScript(ParsedScript scrip
*** 525,530 ****
--- 540,552 ----
static void *threadRun(void *arg);
static void setalarm(int seconds);
static void finishCon(CState *st);
+ static socket_set *alloc_socket_set(int count);
+ static bool error_on_socket(socket_set *sa, int idx, PGconn *con);
+ static void free_socket_set(socket_set *sa);
+ static bool ignore_socket(socket_set *sa, int idx, PGconn *con);
+ static void clear_socket_set(socket_set *sa, int count);
+ static void set_socket(socket_set *sa, int fd, int idx);
+ static int wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec);
/* callback functions for our flex lexer */
*************** doConnect(void)
*** 1143,1148 ****
--- 1165,1171 ----
!have_password)
{
PQfinish(conn);
+ conn = NULL;
simple_prompt("Password: ", password, sizeof(password), false);
have_password = true;
new_pass = true;
*************** main(int argc, char **argv)
*** 4903,4909 ****
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
! if (nclients <= 0 || nclients > MAXCLIENTS)
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
--- 4926,4932 ----
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
! if (nclients <= 0 || (MAXCLIENTS != -1 && nclients > MAXCLIENTS))
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
*************** threadRun(void *arg)
*** 5614,5619 ****
--- 5637,5643 ----
int64 next_report = last_report + (int64) progress * 1000000;
StatsData last,
aggs;
+ socket_set *sockets = alloc_socket_set(nstate);
/*
* Initialize throttling rate target for all of the thread's clients. It
*************** threadRun(void *arg)
*** 5657,5662 ****
--- 5681,5687 ----
{
if ((state[i].con = doConnect()) == NULL)
goto done;
+ set_socket(sockets, PQsocket(state[i].con), i);
}
}
*************** threadRun(void *arg)
*** 5673,5685 ****
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
! FD_ZERO(&input_mask);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
--- 5698,5709 ----
/* loop till all clients have terminated */
while (remains > 0)
{
int maxsock; /* max socket number to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
/* identify which client sockets should be checked for input */
! clear_socket_set(sockets, nstate);
maxsock = -1;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
*************** threadRun(void *arg)
*** 5728,5734 ****
goto done;
}
! FD_SET(sock, &input_mask);
if (maxsock < sock)
maxsock = sock;
}
--- 5752,5758 ----
goto done;
}
! set_socket(sockets, sock, i);
if (maxsock < sock)
maxsock = sock;
}
*************** threadRun(void *arg)
*** 5765,5771 ****
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
! * or it's time to print a progress report. Update input_mask to show
* which client(s) received data.
*/
if (min_usec > 0)
--- 5789,5795 ----
/*
* If no clients are ready to execute actions, sleep until we receive
* data from the server, or a nap-time specified in the script ends,
! * or it's time to print a progress report. Update sockets to show
* which client(s) received data.
*/
if (min_usec > 0)
*************** threadRun(void *arg)
*** 5776,5786 ****
{
if (maxsock != -1)
{
! struct timeval timeout;
!
! timeout.tv_sec = min_usec / 1000000;
! timeout.tv_usec = min_usec % 1000000;
! nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
}
else /* nothing active, simple sleep */
{
--- 5800,5806 ----
{
if (maxsock != -1)
{
! nsocks = wait_on_socket_set(sockets, nstate, maxsock, min_usec);
}
else /* nothing active, simple sleep */
{
*************** threadRun(void *arg)
*** 5789,5795 ****
}
else /* no explicit delay, select without timeout */
{
! nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
}
if (nsocks < 0)
--- 5809,5815 ----
}
else /* no explicit delay, select without timeout */
{
! nsocks = wait_on_socket_set(sockets, nstate, maxsock, 0);
}
if (nsocks < 0)
*************** threadRun(void *arg)
*** 5800,5806 ****
continue;
}
/* must be something wrong */
! fprintf(stderr, "select() failed: %s\n", strerror(errno));
goto done;
}
}
--- 5820,5826 ----
continue;
}
/* must be something wrong */
! fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
*************** threadRun(void *arg)
*** 5809,5815 ****
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
! FD_ZERO(&input_mask);
}
/* ok, advance the state machine of each connection */
--- 5829,5835 ----
/* min_usec == 0, i.e. something needs to be executed */
/* If we didn't call select(), don't try to read any data */
! clear_socket_set(sockets, nstate);
}
/* ok, advance the state machine of each connection */
*************** threadRun(void *arg)
*** 5820,5835 ****
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
- int sock = PQsocket(st->con);
! if (sock < 0)
! {
! fprintf(stderr, "invalid socket: %s",
! PQerrorMessage(st->con));
goto done;
- }
! if (!FD_ISSET(sock, &input_mask))
continue;
}
else if (st->state == CSTATE_FINISHED ||
--- 5840,5850 ----
if (st->state == CSTATE_WAIT_RESULT)
{
/* don't call doCustom unless data is available */
! if (error_on_socket(sockets, i, st->con))
goto done;
! if (ignore_socket(sockets, i, st->con))
continue;
}
else if (st->state == CSTATE_FINISHED ||
*************** done:
*** 5967,5972 ****
--- 5982,5989 ----
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
+ sockets = NULL;
return NULL;
}
*************** finishCon(CState *st)
*** 5980,5985 ****
--- 5997,6131 ----
}
}
+ #ifdef POLL_USING_SELECT /* select(2) based socket polling */
+ static socket_set *
+ alloc_socket_set(int count)
+ {
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+ }
+
+ static void
+ free_socket_set(socket_set *sa)
+ {
+ pg_free(sa);
+ }
+
+ static bool
+ error_on_socket(socket_set *sa, int idx, PGconn *con)
+ {
+ if (PQsocket(con) >= 0) return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ return true;
+ }
+
+ static bool
+ ignore_socket(socket_set *sa, int idx, PGconn *con)
+ {
+ return !(FD_ISSET(PQsocket(con), sa));
+ }
+
+ static void
+ clear_socket_set(socket_set *sa, int count)
+ {
+ FD_ZERO(sa);
+ }
+
+ static void
+ set_socket(socket_set *sa, int fd, int idx)
+ {
+ FD_SET(fd, sa);
+ }
+
+ static int
+ wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+ {
+ struct timeval timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_usec = usec % 1000000;
+ return select(maxsock + 1, sa, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(maxsock + 1, sa, NULL, NULL, NULL);
+ }
+ }
+ #else /* ppoll(2) based socket polling */
+ /* ppoll() will block until timeout or one of POLL_EVENTS occurs. */
+ #define POLL_EVENTS (POLLRDHUP|POLLIN|POLLPRI)
+ /* ppoll() events returned that we do not want/expect to see. */
+ #define POLL_UNWANTED (POLLRDHUP|POLLERR|POLLHUP|POLLNVAL)
+
+ static socket_set *
+ alloc_socket_set(int count)
+ {
+ return (socket_set *) pg_malloc0(sizeof(socket_set) * count);
+ }
+
+ static void
+ free_socket_set(socket_set *sa)
+ {
+ pg_free(sa);
+ }
+
+ static bool
+ error_on_socket(socket_set *sa, int idx, PGconn *con)
+ {
+ /*
+ * No error if socket not used or non-error status from PQsocket() and none
+ * of the unwanted ppoll() return events.
+ */
+ if (sa[idx].fd == -1 || (PQsocket(con) >= 0 && !(sa[idx].revents & POLL_UNWANTED)))
+ return false;
+ fprintf(stderr, "invalid socket: %s", PQerrorMessage(con));
+ if (debug)
+ fprintf(stderr, "ppoll() fail - errno: %d, socket: %d, events: %x\n",
+ errno, sa[idx].fd, (sa[idx].revents & POLL_UNWANTED));
+ return true;
+ }
+
+ static bool
+ ignore_socket(socket_set *sa, int idx, PGconn *con)
+ {
+ return (sa[idx].fd != -1 && !sa[idx].revents);
+ }
+
+ static void
+ clear_socket_set(socket_set *sa, int count)
+ {
+ int i = 0;
+ for (i = 0; i < count; i++)
+ set_socket(sa, -1, i);
+ }
+
+ static void
+ set_socket(socket_set *sa, int fd, int idx)
+ {
+ sa[idx].fd = fd;
+ sa[idx].events = POLL_EVENTS;
+ sa[idx].revents = 0;
+ }
+
+ static int
+ wait_on_socket_set(socket_set *sa, int nstate, int maxsock, int64 usec)
+ {
+ struct timespec timeout;
+
+ if (usec)
+ {
+ timeout.tv_sec = usec / 1000000;
+ timeout.tv_nsec = usec % 1000000000;
+ return ppoll(sa, nstate, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa, nstate, NULL, NULL);
+ }
+ }
+ #endif /* PGBENCH_USE_SELECT */
+
/*
* Support for duration option: set timer_exceeded after so many seconds.
*/
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 4094e22..5d40796 100644
*** a/src/include/pg_config.h.in
--- b/src/include/pg_config.h.in
***************
*** 443,448 ****
--- 443,451 ----
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+ /* Define to 1 if you have the `ppoll' function. */
+ #undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
diff --git a/src/include/pg_config.h.win32 b/src/include/pg_config.h.win32
index 6618b43..182698a 100644
*** a/src/include/pg_config.h.win32
--- b/src/include/pg_config.h.win32
***************
*** 327,332 ****
--- 327,335 ----
/* Define to 1 if you have the `posix_fallocate' function. */
/* #undef HAVE_POSIX_FALLOCATE */
+ /* Define to 1 if you have the `ppoll' function. */
+ /* #undef HAVE_PPOLL */
+
/* Define to 1 if you have the `pstat' function. */
/* #undef HAVE_PSTAT */
diff --git a/src/template/linux b/src/template/linux
index f820bf7..e392908 100644
*** a/src/template/linux
--- b/src/template/linux
*************** if test x"$PREFERRED_SEMAPHORES" = x"" ;
*** 6,11 ****
--- 6,12 ----
fi
# Force _GNU_SOURCE on; plperl is broken with Perl 5.8.0 otherwise
+ # This is also required for ppoll(2), and perhaps other things
CPPFLAGS="$CPPFLAGS -D_GNU_SOURCE"
# If --enable-profiling is specified, we need -DLINUX_PROFILE
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-01 08:34 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-05-17 05:23 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-15 13:59 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Fabien COELHO <coelho@cri.ensmp.fr>
2018-09-22 16:09 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
@ 2018-09-23 00:11 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 51+ messages in thread
From: Tom Lane @ 2018-09-23 00:11 UTC (permalink / raw)
To: Fabien COELHO <coelho@cri.ensmp.fr>; +Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>; Thomas Munro <thomas.munro@enterprisedb.com>; Rady, Doug <radydoug@amazon.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
I wrote:
> I'm strongly tempted to just remove the POLL_UNWANTED business
> altogether, as it seems both pointless and unportable on its face.
> Almost by definition, we can't know what "other" bits a given
> implementation might set.
> I'm not entirely following the point of including POLLRDHUP in
> POLL_EVENTS, either. What's wrong with the traditional solution
> of detecting EOF?
So after studying that a bit longer, I think it's just wrong.
It's not the business of this code to be checking for connection
errors at all; that is libpq's province. The libpq API specifies
that callers should wait for read-ready on the socket, and nothing
else. So the only bit we need concern ourselves with is POLLIN.
I also seriously disliked both the details of the abstraction API
and its lack of documentation. (Other people complained about that
upthread, too.) So attached is a rewrite attempt. There's still a
couple of grotty things about it; in particular the ppoll variant of
socket_has_input() knows more than one could wish about how it's being
used. But I couldn't see a way to make it cleaner without significant
changes to the logic in threadRun, and that didn't seem better.
I think that Andres' concern upthread about iterating over a whole
lot of sockets is somewhat misplaced. We aren't going to be iterating
over the entire set of client connections, only those being run by a
particular pgbench thread. So assuming you're using a reasonable ratio
of threads to clients, there won't be very many to look at in any one
thread. In any case, I'm dubious that we could get much of a win from
some other abstraction for waiting: both of these code paths do work
pretty much proportional to the number of connections the current
thread is responsible for, and it's hard to see how to avoid that.
I've tested this on both Linux and FreeBSD, and it seems to work fine.
I'm reasonably happy with this version of the patch, and would be
ready to commit it, but I thought I'd throw it out for another round
of review if anyone wants to.
regards, tom lane
Attachments:
pgbench-ppoll-17.patch (text/x-diff, 14.0K) download
diff --git a/configure b/configure
index 9b30402..21ecd29 100755
--- a/configure
+++ b/configure
@@ -15093,7 +15093,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 2e60a89..8fe6894 100644
--- a/configure.in
+++ b/configure.in
@@ -1562,7 +1562,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate ppoll pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 41b756c..ae81aba 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -28,8 +28,8 @@
*/
#ifdef WIN32
-#define FD_SETSIZE 1024 /* set before winsock2.h is included */
-#endif /* ! WIN32 */
+#define FD_SETSIZE 1024 /* must set before winsock2.h is included */
+#endif
#include "postgres_fe.h"
#include "fe_utils/conditional.h"
@@ -45,12 +45,21 @@
#include <signal.h>
#include <time.h>
#include <sys/time.h>
+#ifdef HAVE_SYS_RESOURCE_H
+#include <sys/resource.h> /* for getrlimit */
+#endif
+
+/* For testing, PGBENCH_USE_SELECT can be defined to force use of that code */
+#if defined(HAVE_PPOLL) && !defined(PGBENCH_USE_SELECT)
+#define POLL_USING_PPOLL
+#ifdef HAVE_POLL_H
+#include <poll.h>
+#endif
+#else /* no ppoll(), so use select() */
+#define POLL_USING_SELECT
#ifdef HAVE_SYS_SELECT_H
#include <sys/select.h>
#endif
-
-#ifdef HAVE_SYS_RESOURCE_H
-#include <sys/resource.h> /* for getrlimit */
#endif
#ifndef M_PI
@@ -71,6 +80,33 @@
#define MM2_ROT 47
/*
+ * Multi-platform socket set implementations
+ */
+
+#ifdef POLL_USING_PPOLL
+#define SOCKET_WAIT_METHOD "ppoll"
+
+typedef struct socket_set
+{
+ int maxfds; /* allocated length of pollfds[] array */
+ int curfds; /* number currently in use */
+ struct pollfd pollfds[FLEXIBLE_ARRAY_MEMBER];
+} socket_set;
+
+#endif /* POLL_USING_PPOLL */
+
+#ifdef POLL_USING_SELECT
+#define SOCKET_WAIT_METHOD "select"
+
+typedef struct socket_set
+{
+ int maxfd; /* largest FD currently set in fds */
+ fd_set fds;
+} socket_set;
+
+#endif /* POLL_USING_SELECT */
+
+/*
* Multi-platform pthread implementations
*/
@@ -93,13 +129,6 @@ static int pthread_join(pthread_t th, void **thread_return);
/********************************************************************
* some configurable parameters */
-/* max number of clients allowed */
-#ifdef FD_SETSIZE
-#define MAXCLIENTS (FD_SETSIZE - 10)
-#else
-#define MAXCLIENTS 1024
-#endif
-
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
@@ -523,8 +552,14 @@ static void processXactStats(TState *thread, CState *st, instr_time *now,
static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
-static void setalarm(int seconds);
static void finishCon(CState *st);
+static void setalarm(int seconds);
+static socket_set *alloc_socket_set(int count);
+static void free_socket_set(socket_set *sa);
+static void clear_socket_set(socket_set *sa);
+static void add_socket_to_set(socket_set *sa, int fd, int idx);
+static int wait_on_socket_set(socket_set *sa, int64 usecs);
+static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback functions for our flex lexer */
@@ -4903,7 +4938,7 @@ main(int argc, char **argv)
case 'c':
benchmarking_option_set = true;
nclients = atoi(optarg);
- if (nclients <= 0 || nclients > MAXCLIENTS)
+ if (nclients <= 0)
{
fprintf(stderr, "invalid number of clients: \"%s\"\n",
optarg);
@@ -5606,6 +5641,7 @@ threadRun(void *arg)
end;
int nstate = thread->nstate;
int remains = nstate; /* number of remaining clients */
+ socket_set *sockets = alloc_socket_set(nstate);
int i;
/* for reporting progress: */
@@ -5673,14 +5709,16 @@ threadRun(void *arg)
/* loop till all clients have terminated */
while (remains > 0)
{
- fd_set input_mask;
- int maxsock; /* max socket number to be waited for */
+ int nsocks; /* number of sockets to be waited for */
int64 min_usec;
int64 now_usec = 0; /* set this only if needed */
- /* identify which client sockets should be checked for input */
- FD_ZERO(&input_mask);
- maxsock = -1;
+ /*
+ * identify which client sockets should be checked for input, and
+ * compute the nearest time (if any) at which we need to wake up.
+ */
+ clear_socket_set(sockets);
+ nsocks = 0;
min_usec = PG_INT64_MAX;
for (i = 0; i < nstate; i++)
{
@@ -5728,9 +5766,7 @@ threadRun(void *arg)
goto done;
}
- FD_SET(sock, &input_mask);
- if (maxsock < sock)
- maxsock = sock;
+ add_socket_to_set(sockets, sock, nsocks++);
}
else if (st->state != CSTATE_ABORTED &&
st->state != CSTATE_FINISHED)
@@ -5764,35 +5800,29 @@ threadRun(void *arg)
/*
* If no clients are ready to execute actions, sleep until we receive
- * data from the server, or a nap-time specified in the script ends,
- * or it's time to print a progress report. Update input_mask to show
- * which client(s) received data.
+ * data on some client socket or the timeout (if any) elapses.
*/
if (min_usec > 0)
{
- int nsocks = 0; /* return from select(2) if called */
+ int rc = 0;
if (min_usec != PG_INT64_MAX)
{
- if (maxsock != -1)
+ if (nsocks > 0)
{
- struct timeval timeout;
-
- timeout.tv_sec = min_usec / 1000000;
- timeout.tv_usec = min_usec % 1000000;
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, &timeout);
+ rc = wait_on_socket_set(sockets, min_usec);
}
else /* nothing active, simple sleep */
{
pg_usleep(min_usec);
}
}
- else /* no explicit delay, select without timeout */
+ else /* no explicit delay, wait without timeout */
{
- nsocks = select(maxsock + 1, &input_mask, NULL, NULL, NULL);
+ rc = wait_on_socket_set(sockets, 0);
}
- if (nsocks < 0)
+ if (rc < 0)
{
if (errno == EINTR)
{
@@ -5800,19 +5830,20 @@ threadRun(void *arg)
continue;
}
/* must be something wrong */
- fprintf(stderr, "select() failed: %s\n", strerror(errno));
+ fprintf(stderr, "%s() failed: %s\n", SOCKET_WAIT_METHOD, strerror(errno));
goto done;
}
}
else
{
- /* min_usec == 0, i.e. something needs to be executed */
+ /* min_usec <= 0, i.e. something needs to be executed now */
- /* If we didn't call select(), don't try to read any data */
- FD_ZERO(&input_mask);
+ /* If we didn't wait, don't try to read any data */
+ clear_socket_set(sockets);
}
/* ok, advance the state machine of each connection */
+ nsocks = 0;
for (i = 0; i < nstate; i++)
{
CState *st = &state[i];
@@ -5829,7 +5860,7 @@ threadRun(void *arg)
goto done;
}
- if (!FD_ISSET(sock, &input_mask))
+ if (!socket_has_input(sockets, sock, nsocks++))
continue;
}
else if (st->state == CSTATE_FINISHED ||
@@ -5967,6 +5998,7 @@ done:
fclose(thread->logfile);
thread->logfile = NULL;
}
+ free_socket_set(sockets);
return NULL;
}
@@ -6025,8 +6057,185 @@ setalarm(int seconds)
}
}
+#endif /* WIN32 */
+
+
+/*
+ * These functions provide an abstraction layer that hides the syscall
+ * we use to wait for input on a set of sockets.
+ *
+ * Currently there are two implementations, based on ppoll(2) and select(2).
+ * ppoll() is preferred where available due to its typically higher ceiling
+ * on the number of usable sockets. We do not use the more-widely-available
+ * poll(2) because it only offers millisecond timeout resolution, which could
+ * be problematic with high --rate settings.
+ *
+ * Function APIs:
+ *
+ * alloc_socket_set: allocate an empty socket set with room for up to
+ * "count" sockets.
+ *
+ * free_socket_set: deallocate a socket set.
+ *
+ * clear_socket_set: reset a socket set to empty.
+ *
+ * add_socket_to_set: add socket with indicated FD to slot "idx" in the
+ * socket set. Slots must be filled in order, starting with 0.
+ *
+ * wait_on_socket_set: wait for input on any socket in set, or for timeout
+ * to expire. timeout is measured in microseconds; 0 means wait forever.
+ * Returns result code of underlying syscall (>=0 if OK, else see errno).
+ *
+ * socket_has_input: after waiting, call this to see if given socket has
+ * input. fd and idx parameters should match some previous call to
+ * add_socket_to_set.
+ *
+ * Note that wait_on_socket_set destructively modifies the state of the
+ * socket set. After checking for input, caller must apply clear_socket_set
+ * and add_socket_to_set again before waiting again.
+ */
+
+#ifdef POLL_USING_PPOLL
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ socket_set *sa;
+
+ sa = (socket_set *) pg_malloc0(offsetof(socket_set, pollfds) +
+ sizeof(struct pollfd) * count);
+ sa->maxfds = count;
+ sa->curfds = 0;
+ return sa;
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static void
+clear_socket_set(socket_set *sa)
+{
+ sa->curfds = 0;
+}
+
+static void
+add_socket_to_set(socket_set *sa, int fd, int idx)
+{
+ Assert(idx < sa->maxfds && idx == sa->curfds);
+ sa->pollfds[idx].fd = fd;
+ sa->pollfds[idx].events = POLLIN;
+ sa->pollfds[idx].revents = 0;
+ sa->curfds++;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int64 usecs)
+{
+ if (usecs > 0)
+ {
+ struct timespec timeout;
+
+ timeout.tv_sec = usecs / 1000000;
+ timeout.tv_nsec = (usecs % 1000000) * 1000;
+ return ppoll(sa->pollfds, sa->curfds, &timeout, NULL);
+ }
+ else
+ {
+ return ppoll(sa->pollfds, sa->curfds, NULL, NULL);
+ }
+}
+
+static bool
+socket_has_input(socket_set *sa, int fd, int idx)
+{
+ /*
+ * In some cases, threadRun will apply clear_socket_set and then try to
+ * apply socket_has_input anyway with arguments that it used before that,
+ * or might've used before that except that it exited its setup loop
+ * early. Hence, if the socket set is empty, silently return false
+ * regardless of the parameters. If it's not empty, we can Assert that
+ * the parameters match a previous call.
+ */
+ if (sa->curfds == 0)
+ return false;
+
+ Assert(idx < sa->curfds && sa->pollfds[idx].fd == fd);
+ return (sa->pollfds[idx].revents & POLLIN) != 0;
+}
+
+#endif /* POLL_USING_PPOLL */
+
+#ifdef POLL_USING_SELECT
+
+static socket_set *
+alloc_socket_set(int count)
+{
+ return (socket_set *) pg_malloc0(sizeof(socket_set));
+}
+
+static void
+free_socket_set(socket_set *sa)
+{
+ pg_free(sa);
+}
+
+static void
+clear_socket_set(socket_set *sa)
+{
+ FD_ZERO(&sa->fds);
+ sa->maxfd = -1;
+}
+
+static void
+add_socket_to_set(socket_set *sa, int fd, int idx)
+{
+ if (fd < 0 || fd >= FD_SETSIZE)
+ {
+ /*
+ * Doing a hard exit here is a bit grotty, but it doesn't seem worth
+ * complicating the API to make it less grotty.
+ */
+ fprintf(stderr, "too many client connections for select()\n");
+ exit(1);
+ }
+ FD_SET(fd, &sa->fds);
+ if (fd > sa->maxfd)
+ sa->maxfd = fd;
+}
+
+static int
+wait_on_socket_set(socket_set *sa, int64 usecs)
+{
+ if (usecs > 0)
+ {
+ struct timeval timeout;
+
+ timeout.tv_sec = usecs / 1000000;
+ timeout.tv_usec = usecs % 1000000;
+ return select(sa->maxfd + 1, &sa->fds, NULL, NULL, &timeout);
+ }
+ else
+ {
+ return select(sa->maxfd + 1, &sa->fds, NULL, NULL, NULL);
+ }
+}
+
+static bool
+socket_has_input(socket_set *sa, int fd, int idx)
+{
+ return (FD_ISSET(fd, &sa->fds) != 0);
+}
+
+#endif /* POLL_USING_SELECT */
+
+
/* partial pthread implementation for Windows */
+#ifdef WIN32
+
typedef struct win32_pthread
{
HANDLE handle;
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 4094e22..5d40796 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -443,6 +443,9 @@
/* Define to 1 if the assembler supports PPC's LWARX mutex hint bit. */
#undef HAVE_PPC_LWARX_MUTEX_HINT
+/* Define to 1 if you have the `ppoll' function. */
+#undef HAVE_PPOLL
+
/* Define to 1 if you have the `pstat' function. */
#undef HAVE_PSTAT
diff --git a/src/include/pg_config.h.win32 b/src/include/pg_config.h.win32
index 6618b43..182698a 100644
--- a/src/include/pg_config.h.win32
+++ b/src/include/pg_config.h.win32
@@ -327,6 +327,9 @@
/* Define to 1 if you have the `posix_fallocate' function. */
/* #undef HAVE_POSIX_FALLOCATE */
+/* Define to 1 if you have the `ppoll' function. */
+/* #undef HAVE_PPOLL */
+
/* Define to 1 if you have the `pstat' function. */
/* #undef HAVE_PSTAT */
diff --git a/src/template/linux b/src/template/linux
index f820bf7..e392908 100644
--- a/src/template/linux
+++ b/src/template/linux
@@ -6,6 +6,7 @@ if test x"$PREFERRED_SEMAPHORES" = x"" ; then
fi
# Force _GNU_SOURCE on; plperl is broken with Perl 5.8.0 otherwise
+# This is also required for ppoll(2), and perhaps other things
CPPFLAGS="$CPPFLAGS -D_GNU_SOURCE"
# If --enable-profiling is specified, we need -DLINUX_PROFILE
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
@ 2018-09-19 21:46 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-21 15:44 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 51+ messages in thread
From: Tom Lane @ 2018-09-19 21:46 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
"Rady, Doug" <radydoug@amazon.com> writes:
> This patch enables building pgbench to use ppoll() instead of select()
> to allow for more than (FD_SETSIZE - 10) connections. As implemented,
> when using ppoll(), the only connection limitation is system resources.
So ... why exactly is this patch insisting on ppoll() rather than just
plain poll()? AFAICS, all you're doing with that is being able to
specify the timeout in microsec not millisec, which does not really
justify taking much of a hit in portability, to my mind.
> “… ppoll() is to poll() as pselect() is to select(). Since the
> existing code uses select(), why not convert to poll() rather than
> ppoll()?”
A moment's glance at our git history will remind you that we attempted
to start using pselect() last year, and the attempt crashed and burned:
Author: Tom Lane <tgl@sss.pgh.pa.us>
Branch: master Release: REL_10_BR [64925603c] 2017-04-24 18:29:03 -0400
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop."
This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4.
Buildfarm results suggest that some platforms have versions of pselect(2)
that are not merely non-atomic, but flat out non-functional. Revert the
use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART
patch as the source of trouble). If it's so, we should probably look into
blacklisting specific platforms that have broken pselect.
Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
Now, it might be that ppoll doesn't suffer from as many portability
problems as pselect, but I don't see a good reason to assume that.
So I'd rather see if we can't convert this to use poll(), and thereby
ensure that it works basically everywhere.
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
* Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-09-19 21:46 ` Re: PATCH: pgbench - option to build using ppoll() for larger connection counts Tom Lane <tgl@sss.pgh.pa.us>
@ 2018-09-21 15:44 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 51+ messages in thread
From: Tom Lane @ 2018-09-21 15:44 UTC (permalink / raw)
To: Rady, Doug <radydoug@amazon.com>; +Cc: pgsql-hackers; Robert Haas <robertmhaas@gmail.com>
I wrote:
> So ... why exactly is this patch insisting on ppoll() rather than just
> plain poll()? AFAICS, all you're doing with that is being able to
> specify the timeout in microsec not millisec, which does not really
> justify taking much of a hit in portability, to my mind.
To check into my assumptions here, I did a bit of testing to find out
what ppoll can really do in terms of timeout accuracy. As best I can
tell, its resolution is on the order of 100 usec on both fairly new
Linux kernels (4.17.19 on Fedora 28) and fairly old ones (2.6.32 on
RHEL6). So there's not really that much daylight between what you can
do with ppoll() and with poll()'s millisecond resolution. select(2)
seems to have about the same effective timeout resolution.
Interestingly, select(2)'s timeout resolution is noticeably better than
that, around 10 usec, on recent macOS. Didn't try other platforms.
Also, I'd misunderstood the portability angle. Unlike pselect(),
ppoll() is *not* in POSIX. It started as a Linux-ism, although
I see it's appeared recently in FreeBSD. That somewhat assuages
my fear of broken implementations on specific platforms, but it
also means that it's going to be far from universally available.
So the question here really boils down to whether you think that a
large set of pgbench connections is interesting on non-Linux platforms.
There's a case to be made that it isn't, perhaps, but I'm not exactly
sold.
On the other hand, while we could certainly make a poll-based code path
within this patch, I'm not quite sure what we'd do with it. My results
for macOS indicate that using poll rather than select would create a
tradeoff: in return for possibly allowing more clients, there would be
a definite loss in timeout precision. I don't think that limiting
\sleep commands to ms precision would be so awful, but it's easier
to believe that loss of precision in the transaction dispatch rate for
"--rate" tests could be a problem for some people. So we might have
to expose the choice of which call to use to users, which seems like
a mess.
So maybe what we've got here --- make it better on Linux, with no
change elsewhere --- is about as good as we can hope for.
Also, I notice that the kevent syscall available on macOS and some
BSDen uses a struct-timespec timeout parameter, ie, theoretical nsec
resolution same as ppoll. So there's a case to be made that where
we should go for those platforms is to build a kevent-based code
path not a poll-based one. It'd be unreasonable to insist on this
patch providing that, though.
Anyway, bottom line: my objection on Wednesday was mainly based on
the assumption that we might have to contend with broken ppoll on
some platforms, which now appears to be fallacious. So, objection
withdrawn.
regards, tom lane
^ permalink raw reply [nested|flat] 51+ messages in thread
end of thread, other threads:[~2018-09-23 00:11 UTC | newest]
Thread overview: 51+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-09-25 18:01 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-09-25 18:07 ` Andres Freund <andres@anarazel.de>
2017-09-25 20:40 ` Rady, Doug <radydoug@amazon.com>
2017-09-25 18:10 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-04 06:20 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-23 18:44 ` Rady, Doug <radydoug@amazon.com>
2017-11-03 05:29 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-28 01:41 ` Michael Paquier <michael.paquier@gmail.com>
2017-11-28 05:39 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-10-26 11:45 ` Robert Haas <robertmhaas@gmail.com>
2017-10-26 13:21 ` Rady, Doug <radydoug@amazon.com>
2017-09-25 22:29 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-28 15:38 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2017-11-28 18:05 ` Robert Haas <robertmhaas@gmail.com>
2017-11-29 08:40 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 12:58 ` Robert Haas <robertmhaas@gmail.com>
2017-11-29 13:10 ` Fabien COELHO <coelho@cri.ensmp.fr>
2017-11-29 14:15 ` Robert Haas <robertmhaas@gmail.com>
2017-11-29 14:26 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-06 21:49 ` Stephen Frost <sfrost@snowman.net>
2018-01-22 08:34 PATCH: pgbench - option to build using ppoll() for larger connection counts Rady, Doug <radydoug@amazon.com>
2018-01-24 08:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:05 ` Rady, Doug <radydoug@amazon.com>
2018-01-25 15:16 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-25 15:34 ` Rady, Doug <radydoug@amazon.com>
2018-01-25 22:46 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-26 22:27 ` Rady, Doug <radydoug@amazon.com>
2018-01-26 23:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-01-28 23:02 ` Rady, Doug <radydoug@amazon.com>
2018-01-29 08:58 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-01 08:34 ` Andres Freund <andres@anarazel.de>
2018-03-01 10:30 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-04 00:14 ` Andres Freund <andres@anarazel.de>
2018-03-21 05:42 ` Rady, Doug <radydoug@amazon.com>
2018-03-25 11:00 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-03-26 20:23 ` Rady, Doug <radydoug@amazon.com>
2018-03-27 09:06 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-04-06 21:15 ` Andres Freund <andres@anarazel.de>
2018-04-06 21:49 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-04-06 21:54 ` Andres Freund <andres@anarazel.de>
2018-04-07 05:20 ` konstantin knizhnik <k.knizhnik@postgrespro.ru>
2018-05-17 05:23 ` Thomas Munro <thomas.munro@enterprisedb.com>
2018-07-03 23:52 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 16:45 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-09 21:46 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-08-10 01:32 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-15 13:59 ` Fabien COELHO <coelho@cri.ensmp.fr>
2018-09-22 16:09 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-23 00:11 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-19 21:46 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-21 15:44 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox