public inbox for [email protected]
help / color / mirror / Atom feedFrom: Nadav Shatz <[email protected]>
To: Tatsuo Ishii <[email protected]>
Cc: [email protected]
Subject: Re: Proposal: recent access based routing for primary-replica setups
Date: Sun, 24 Aug 2025 14:11:32 +0300
Message-ID: <CACeKOO000N8gUodkXt_1V=XQYx_utxXdyOxkzV7GYSPpdHnTfg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CACeKOO2t9hhwr82ksziBApBGZtEUn9KuXqS-bJB-=N7KRs6Acg@mail.gmail.com>
<[email protected]>
<CACeKOO3KF9ZdhkvgM+znSaj6dkTyMR77xCebQk9iuTfVavkYbA@mail.gmail.com>
<[email protected]>
Hi Tatsuo,
Here is an initial draft in 2 patches (one for code changes and one for
tests implementation).
Please let me know what you think.
Thank you,
On Thu, Aug 21, 2025 at 1:23 PM Tatsuo Ishii <[email protected]> wrote:
> Hi Nadav,
>
> Thank you for understanding. Please don't hesitate to ask questions
> regarding Pgpool-II source code.
>
> Best regards,
> --
> Tatsuo Ishii
> SRA OSS K.K.
> English: http://www.sraoss.co.jp/index_en/
> Japanese:http://www.sraoss.co.jp
>
> > Hi Tatsuo,
> >
> > I'm fine with all of your comments and suggestions.
> >
> > I'll work on a patch and we can iterate over it.
> >
> > Hope that's okay.
> >
> > Best,
> >
> > On Thu, Aug 21, 2025 at 8:04 AM Tatsuo Ishii <[email protected]>
> wrote:
> >
> >> Hi Nadav,
> >>
> >> > Hi Tatsuo,
> >> >
> >> > Thank you for your reply, I agree with your approach. Better to get
> (1)
> >> out
> >> > of the way first.
> >> >
> >> > As a simplest approach that we can implement that would support
> >> completely
> >> > offloading the responsibility of the lag checking we can set it to
> “file”
> >> > and add another config for file path. Or just if starts with “file:”
> >> it’ll
> >> > understand.
> >>
> >> My concern about the "file:" approach is, race condition. What if
> >> pgpool reads the file while it is being updated by someone else? Also
> >> I think the command approach is more flexible and generic. For
> >> example, the "file approch" can be easily simulated by setting the
> >> command "/usr/bin/cat path_to_the_file".
> >>
> >> > Then the internal polling can just read the file on schedule. The
> entire
> >> > updating mechanism will be left to the external service.
> >>
> >> Internal polling is a little bit complicated and will not be easily
> >> changed to just reading a file. The internal polling has two options:
> >> one is checking WAL LSN difference, the other is replication delay in
> >> time. The file approch would only replace the latter. I suggest to
> >> leave the internal polling code as it is.
> >>
> >> > Having this as a first step also opens up the door for other
> >> > implementations.
> >> >
> >> > Another classic option would be calling an API endpoint. But that
> might
> >> > come with a lot more bulk and security concerns.
> >>
> >> I agree that calling API could bring security concerns.
> >>
> >> BTW, in the command approch, the command should be executed as
> >> sr_check_user.
> >>
> >> > I suggest I work on a patch for file support.
> >> >
> >> > What do you think?
> >>
> >> For the reason above I prefer the command approch, not the file
> >> support.
> >>
> >> > Nadav Shatz
> >> > Tailor Brands | CTO
> >> >
> >> >
> >> > On Wed, Aug 20, 2025 at 3:45 PM Tatsuo Ishii <[email protected]>
> >> wrote:
> >> >
> >> >> Hi Nadav,
> >> >>
> >> >> Thank you for the answer.
> >> >>
> >> >> I think your proposal actually includes two orthogonal proposals.
> >> >>
> >> >> (1) "inject" replication delay value from external source (in your
> >> >> case from Aurora).
> >> >>
> >> >> (2) per relation recent access based routing.
> >> >>
> >> >> I suggest to implement (1) first, then (2). This incremental approach
> >> >> would be easier than implementing (1)+(2) at once.
> >> >>
> >> >> For (1) we could add new pgpool.conf parameter, say
> >> >> "replication_delay_source". If it is set to "builtin", then
> >> >> replication delay source is PostgreSQL as we already does today. If
> >> >> it's set other than "builtin", then it's an external command name (+
> >> >> arguments) to be executed to import replication delay value. The
> >> >> command should return replication delay value represented in strings
> >> >> like "0 20 10", which means node 0, 1 and 2 replication delay values
> >> >> in millisecond (in this case since the node 0 is primary, its
> >> >> replication delay is 0). The command will be invoked every
> >> >> sr_check_period.
> >> >>
> >> >> I am not sure if this actually works in Aurora. This is just a quick
> >> >> idea.
> >> >>
> >> >> (2) would be probably much harder than (1). So we need more
> discussion
> >> >> later on.
> >> >>
> >> >> Best regards,
> >> >> --
> >> >> Tatsuo Ishii
> >> >> SRA OSS K.K.
> >> >> English: http://www.sraoss.co.jp/index_en/
> >> >> Japanese:http://www.sraoss.co.jp
> >> >>
> >>
> >
> >
> > --
> > Nadav Shatz
> > Tailor Brands | CTO
>
--
Nadav Shatz
Tailor Brands | CTO
Attachments:
[application/octet-stream] external-lag-feature-implementation.patch (16.0K, 3-external-lag-feature-implementation.patch)
download | inline diff:
From 6a1ff112ceb5fa1b6b344769ededfafa55eb8d90 Mon Sep 17 00:00:00 2001
From: Nadav Shatz <[email protected]>
Date: Sun, 24 Aug 2025 13:49:36 +0300
Subject: [PATCH] Add external command replication delay source feature
This patch introduces a comprehensive external command replication delay
source feature that allows pgpool to retrieve replication delay information
from external commands instead of built-in database queries.
Key Features:
- External command execution with configurable timeout (1-3600 seconds)
- Secure command construction with injection protection
- Support for running commands as specific users (sr_check_user)
- Comprehensive input validation and error handling
- Graceful fallback to built-in method on failures
Configuration Options:
- replication_delay_source: 'builtin' (default) or 'cmd'
- replication_delay_source_cmd: external command to execute
- replication_delay_source_timeout: command timeout in seconds (default: 10)
Security Features:
- Command injection protection via proper single-quote escaping
- Safe su command construction preventing malicious execution
- Input validation to prevent injection through delay values
- Comprehensive range validation for delay values
Robustness Features:
- SIGALRM-based timeout mechanism with proper signal handling
- Dynamic buffer allocation (4KB) with truncation detection
- PG_TRY/PG_CATCH blocks for proper error handling and cleanup
- Memory leak prevention in all error paths
- Token count validation ensuring output matches NUM_BACKENDS
- Primary node delay correction (always 0ms)
- Support for both integer and floating-point delay values
Command Format:
External commands should output space-separated delay values in milliseconds:
"node0_delay node1_delay node2_delay ..."
Example: "0 25.5 100" (primary: 0ms, standby1: 25.5ms, standby2: 100ms)
This enables integration with custom monitoring solutions, external
replication lag measurement tools, and enterprise monitoring systems
while maintaining full backward compatibility and security.
---
src/config/pool_config_variables.c | 36 ++
src/include/pool_config.h | 9 +
src/sample/pgpool.conf.sample-stream | 16 +
src/streaming_replication/pool_worker_child.c | 333 +++++++++++++++++-
4 files changed, 393 insertions(+), 1 deletion(-)
diff --git a/src/config/pool_config_variables.c b/src/config/pool_config_variables.c
index 5bbe46d3a..233bada89 100644
--- a/src/config/pool_config_variables.c
+++ b/src/config/pool_config_variables.c
@@ -310,6 +310,12 @@ static const struct config_enum_entry check_temp_table_options[] = {
{NULL, 0, false}
};
+static const struct config_enum_entry replication_delay_source_options[] = {
+ {"builtin", REPLICATION_DELAY_BUILTIN, false},
+ {"cmd", REPLICATION_DELAY_CMD, false},
+ {NULL, 0, false}
+};
+
static const struct config_enum_entry log_backend_messages_options[] = {
{"none", BGMSG_NONE, false}, /* turn off logging */
{"terse", BGMSG_TERSE, false}, /* terse logging (repeated messages are
@@ -980,6 +986,36 @@ static struct config_string ConfigureNamesString[] =
NULL, NULL, NULL, NULL
},
+ {
+ {"replication_delay_source", CFGCXT_RELOAD, STREAMING_REPLICATION_CONFIG,
+ "Source of replication delay information.",
+ CONFIG_VAR_TYPE_ENUM, false, 0
+ },
+ &g_pool_config.replication_delay_source,
+ "builtin",
+ NULL, NULL, NULL, replication_delay_source_options
+ },
+
+ {
+ {"replication_delay_source_cmd", CFGCXT_RELOAD, STREAMING_REPLICATION_CONFIG,
+ "External command to retrieve replication delay information.",
+ CONFIG_VAR_TYPE_STRING, false, 0
+ },
+ &g_pool_config.replication_delay_source_cmd,
+ "",
+ NULL, NULL, NULL, NULL
+ },
+
+ {
+ {"replication_delay_source_timeout", CFGCXT_RELOAD, STREAMING_REPLICATION_CONFIG,
+ "Timeout for external replication delay command execution in seconds.",
+ CONFIG_VAR_TYPE_INT, false, 0
+ },
+ &g_pool_config.replication_delay_source_timeout,
+ 10,
+ 1, 3600, NULL, NULL
+ },
+
{
{"failback_command", CFGCXT_RELOAD, FAILOVER_CONFIG,
"Command to execute when backend node is attached.",
diff --git a/src/include/pool_config.h b/src/include/pool_config.h
index be82750e5..1a8262dd7 100644
--- a/src/include/pool_config.h
+++ b/src/include/pool_config.h
@@ -94,6 +94,12 @@ typedef enum LogStandbyDelayModes
LSD_NONE
} LogStandbyDelayModes;
+typedef enum ReplicationDelaySourceModes
+{
+ REPLICATION_DELAY_BUILTIN = 1,
+ REPLICATION_DELAY_CMD
+} ReplicationDelaySourceModes;
+
typedef enum MemCacheMethod
{
@@ -371,6 +377,9 @@ typedef struct
char *sr_check_password; /* password for sr_check_user */
char *sr_check_database; /* PostgreSQL database name for streaming
* replication check */
+ int replication_delay_source; /* replication delay source: builtin or cmd */
+ char *replication_delay_source_cmd; /* external command for replication delay */
+ int replication_delay_source_timeout; /* timeout for external command in seconds */
char *failover_command; /* execute command when failover happens */
char *follow_primary_command; /* execute command when failover is
* ended */
diff --git a/src/sample/pgpool.conf.sample-stream b/src/sample/pgpool.conf.sample-stream
index a7eb594c9..76d51e0fa 100644
--- a/src/sample/pgpool.conf.sample-stream
+++ b/src/sample/pgpool.conf.sample-stream
@@ -519,6 +519,22 @@ backend_clustering_mode = streaming_replication
#sr_check_database = 'postgres'
# Database name for streaming replication check
+
+#replication_delay_source = 'builtin'
+ # Source of replication delay information
+ # 'builtin': use built-in database queries (default)
+ # 'cmd': use external command
+#replication_delay_source_cmd = ''
+ # External command to retrieve replication delay information
+ # Only used when replication_delay_source = 'cmd'
+ # Command should output delay values in milliseconds
+ # Format: "0 20 10" (node0 node1 node2 delays)
+ # Command runs with sr_check_user credentials
+#replication_delay_source_timeout = 10
+ # Timeout for external command execution in seconds
+ # Only used when replication_delay_source = 'cmd'
+ # Range: 1-3600 seconds (default: 10)
+
#delay_threshold = 0
# Threshold before not dispatching query to standby node
# Unit is in bytes
diff --git a/src/streaming_replication/pool_worker_child.c b/src/streaming_replication/pool_worker_child.c
index 4f8f823a3..a80dc27a4 100644
--- a/src/streaming_replication/pool_worker_child.c
+++ b/src/streaming_replication/pool_worker_child.c
@@ -76,6 +76,7 @@ static volatile sig_atomic_t restart_request = 0;
static void establish_persistent_connection(void);
static void discard_persistent_connection(void);
static void check_replication_time_lag(void);
+static void check_replication_time_lag_with_cmd(void);
static void CheckReplicationTimeLagErrorCb(void *arg);
static unsigned long long int text_to_lsn(char *text);
static RETSIGTYPE my_signal_handler(int sig);
@@ -259,7 +260,10 @@ do_worker_child(void)
POOL_NODE_STATUS *node_status;
int i;
- /* Do replication time lag checking */
+ /* Do replication time lag checking */
+ if (pool_config->replication_delay_source == REPLICATION_DELAY_CMD)
+ check_replication_time_lag_with_cmd();
+ else
check_replication_time_lag();
/* Check node status */
@@ -659,6 +663,333 @@ check_replication_time_lag(void)
error_context_stack = callback.previous;
}
+#define MAX_CMD_OUTPUT 4096
+#define MAX_REASONABLE_DELAY_MS 3600000.0 /* 1 hour in milliseconds */
+
+/* Global variable to track command timeout */
+static volatile sig_atomic_t command_timeout_occurred = 0;
+
+/*
+ * Signal handler for command timeout
+ */
+static void
+command_timeout_handler(int sig)
+{
+ command_timeout_occurred = 1;
+}
+
+/*
+ * Escape single quotes in a string for shell command safety
+ */
+static char *
+escape_single_quotes(const char *input)
+{
+ const char *src;
+ char *result, *dst;
+ int quote_count = 0;
+ int len;
+
+ /* Count single quotes to determine result size */
+ for (src = input; *src; src++)
+ {
+ if (*src == '\'')
+ quote_count++;
+ }
+
+ /* Allocate result: original length + 3 chars per quote (replace ' with '\''') */
+ len = strlen(input) + (quote_count * 3) + 1;
+ result = palloc(len);
+
+ /* Copy and escape */
+ dst = result;
+ for (src = input; *src; src++)
+ {
+ if (*src == '\'')
+ {
+ /* Replace ' with '\'' */
+ *dst++ = '\'';
+ *dst++ = '\\';
+ *dst++ = '\'';
+ *dst++ = '\'';
+ }
+ else
+ {
+ *dst++ = *src;
+ }
+ }
+ *dst = '\0';
+
+ return result;
+}
+
+/*
+ * Check replication time lag using external command
+ */
+static void
+check_replication_time_lag_with_cmd(void)
+{
+ FILE *fp;
+ char *command = NULL;
+ char *escaped_cmd = NULL;
+ char *line;
+ char *token;
+ char *saveptr;
+ int node_id;
+ double delay_ms;
+ uint64 delay;
+ int token_count = 0;
+ BackendInfo *bkinfo;
+ ErrorContextCallback callback;
+ bool cmd_allocated = false;
+
+ if (NUM_BACKENDS <= 1)
+ {
+ /* If there's only one node, there's no point to do checking */
+ return;
+ }
+
+ if (REAL_PRIMARY_NODE_ID < 0)
+ {
+ /* No need to check if there's no primary */
+ return;
+ }
+
+ if (!VALID_BACKEND(REAL_PRIMARY_NODE_ID))
+ {
+ /* No need to check replication delay if primary is down */
+ return;
+ }
+
+ if (!pool_config->replication_delay_source_cmd ||
+ strlen(pool_config->replication_delay_source_cmd) == 0)
+ {
+ ereport(WARNING,
+ (errmsg("replication_delay_source is set to 'cmd' but replication_delay_source_cmd is not configured"),
+ errhint("Set replication_delay_source_cmd or change replication_delay_source to 'builtin'")));
+ /* Fall back to builtin method */
+ check_replication_time_lag();
+ return;
+ }
+
+ /* Allocate buffer for command output */
+ line = palloc(MAX_CMD_OUTPUT);
+
+ /*
+ * Register a error context callback to throw proper context message
+ */
+ callback.callback = CheckReplicationTimeLagErrorCb;
+ callback.arg = NULL;
+ callback.previous = error_context_stack;
+ error_context_stack = &callback;
+
+ /* Build command to run as sr_check_user if specified */
+ PG_TRY();
+ {
+ if (pool_config->sr_check_user && strlen(pool_config->sr_check_user) > 0)
+ {
+ char *full_command;
+ int cmd_len;
+
+ /* Escape the command to prevent injection */
+ escaped_cmd = escape_single_quotes(pool_config->replication_delay_source_cmd);
+
+ cmd_len = strlen(escaped_cmd) +
+ strlen(pool_config->sr_check_user) + 20; /* extra space for "su - user -c ''" */
+
+ full_command = palloc(cmd_len);
+ snprintf(full_command, cmd_len, "su - %s -c '%s'",
+ pool_config->sr_check_user, escaped_cmd);
+ command = full_command;
+ cmd_allocated = true;
+ }
+ else
+ {
+ command = pool_config->replication_delay_source_cmd;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("executing replication delay command: %s", command)));
+
+ /* Set up timeout for command execution */
+ command_timeout_occurred = 0;
+ signal(SIGALRM, command_timeout_handler);
+ alarm(pool_config->replication_delay_source_timeout);
+
+ fp = popen(command, "r");
+ if (fp == NULL)
+ {
+ alarm(0); /* Cancel alarm */
+ signal(SIGALRM, SIG_DFL);
+ ereport(ERROR,
+ (errmsg("failed to execute replication delay command: %s", command),
+ errdetail("popen failed: %m")));
+ }
+
+ if (fgets(line, MAX_CMD_OUTPUT, fp) == NULL)
+ {
+ int pclose_result = pclose(fp);
+ fp = NULL;
+ alarm(0); /* Cancel alarm */
+ signal(SIGALRM, SIG_DFL);
+
+ if (command_timeout_occurred)
+ {
+ ereport(ERROR,
+ (errmsg("replication delay command timed out after %d seconds: %s",
+ pool_config->replication_delay_source_timeout, command),
+ errhint("Consider increasing replication_delay_source_timeout or optimizing the command")));
+ }
+ else
+ {
+ ereport(ERROR,
+ (errmsg("failed to read output from replication delay command: %s", command),
+ errdetail("command exit status: %d", pclose_result)));
+ }
+ }
+
+ alarm(0); /* Cancel alarm */
+ signal(SIGALRM, SIG_DFL);
+
+ /* Check if output was truncated */
+ if (strlen(line) == MAX_CMD_OUTPUT - 1 && line[MAX_CMD_OUTPUT - 2] != '\n')
+ {
+ ereport(WARNING,
+ (errmsg("replication delay command output may have been truncated")));
+ }
+
+ pclose(fp);
+ fp = NULL;
+
+ /* Parse the output format "0 20 10" where each number is delay in milliseconds for nodes 0, 1, 2 etc */
+ /* Count tokens first for validation */
+ char *line_copy = pstrdup(line);
+ char *temp_token = strtok(line_copy, " \t\n");
+ while (temp_token != NULL)
+ {
+ token_count++;
+ temp_token = strtok(NULL, " \t\n");
+ }
+ pfree(line_copy);
+
+ /* Now parse the actual tokens */
+ token = strtok_r(line, " \t\n", &saveptr);
+ node_id = 0;
+
+ if (token_count != NUM_BACKENDS)
+ {
+ ereport(WARNING,
+ (errmsg("replication delay command returned %d values, expected %d",
+ token_count, NUM_BACKENDS),
+ errhint("Command should output one delay value per backend node")));
+ }
+
+ while (token != NULL && node_id < NUM_BACKENDS)
+ {
+ if (!VALID_BACKEND(node_id))
+ {
+ node_id++;
+ token = strtok_r(NULL, " \t\n", &saveptr);
+ continue;
+ }
+
+ char *endptr;
+ delay_ms = strtod(token, &endptr);
+
+ /* Validate the conversion */
+ if (*endptr != '\0')
+ {
+ ereport(WARNING,
+ (errmsg("invalid delay value '%s' for node %d, treating as 0",
+ token, node_id)));
+ delay_ms = 0;
+ }
+
+ /* Validate delay value range */
+ if (delay_ms < 0)
+ {
+ ereport(WARNING,
+ (errmsg("negative delay value %.3f for node %d, treating as 0",
+ delay_ms, node_id)));
+ delay_ms = 0;
+ }
+ else if (delay_ms > MAX_REASONABLE_DELAY_MS)
+ {
+ ereport(WARNING,
+ (errmsg("extremely large delay value %.3f for node %d",
+ delay_ms, node_id)));
+ }
+
+ bkinfo = pool_get_node_info(node_id);
+
+ if (PRIMARY_NODE_ID == node_id)
+ {
+ /* Primary node should always have 0 delay */
+ bkinfo->standby_delay = 0;
+ if (delay_ms > 0)
+ {
+ ereport(DEBUG1,
+ (errmsg("primary node %d reported non-zero delay %.3f, setting to 0",
+ node_id, delay_ms)));
+ }
+ }
+ else
+ {
+ /* Convert delay from milliseconds to microseconds for internal storage */
+ delay = (uint64)(delay_ms * 1000);
+ bkinfo->standby_delay = delay;
+ bkinfo->standby_delay_by_time = true;
+
+ /* Log delay if necessary */
+ uint64 delay_threshold_by_time = pool_config->delay_threshold_by_time * 1000; /* threshold is in milliseconds, convert to microseconds */
+
+ if ((pool_config->log_standby_delay == LSD_ALWAYS && delay_ms > 0) ||
+ (pool_config->log_standby_delay == LSD_OVER_THRESHOLD &&
+ bkinfo->standby_delay > delay_threshold_by_time))
+ {
+ ereport(LOG,
+ (errmsg("Replication of node: %d is behind %.3f second(s) from the primary server (node: %d) [external command]",
+ node_id, delay_ms / 1000, PRIMARY_NODE_ID)));
+ }
+ }
+
+ node_id++;
+ token = strtok_r(NULL, " \t\n", &saveptr);
+ }
+
+ }
+ }
+ PG_CATCH();
+ {
+ /* Cleanup in case of error */
+ alarm(0); /* Cancel any pending alarm */
+ signal(SIGALRM, SIG_DFL);
+ if (fp)
+ {
+ pclose(fp);
+ fp = NULL;
+ }
+ if (line)
+ pfree(line);
+ if (escaped_cmd)
+ pfree(escaped_cmd);
+ if (cmd_allocated && command)
+ pfree(command);
+ error_context_stack = callback.previous;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ /* Normal cleanup */
+ if (line)
+ pfree(line);
+ if (escaped_cmd)
+ pfree(escaped_cmd);
+ if (cmd_allocated && command)
+ pfree(command);
+
+ error_context_stack = callback.previous;
+}
+
static void
CheckReplicationTimeLagErrorCb(void *arg)
{
--
2.51.0
[application/octet-stream] external-lag-feature-tests.patch (25.7K, 4-external-lag-feature-tests.patch)
download | inline diff:
From 4d843e69bacb6c1806fe2409a7c304f4fed7f385 Mon Sep 17 00:00:00 2001
From: Nadav Shatz <[email protected]>
Date: Sun, 24 Aug 2025 13:49:54 +0300
Subject: [PATCH] Add comprehensive test suite for external replication delay
feature
This patch adds a complete test suite for the external command replication
delay source feature with comprehensive coverage of all functionality and
edge cases.
Test Coverage:
- Basic external command execution with integer millisecond values
- Floating-point millisecond value parsing and handling
- Delay threshold functionality with external commands
- User switching with sr_check_user parameter
- Error handling for missing/invalid commands and execution failures
- Command timeout handling with configurable timeout values
- Input validation for invalid, negative, and extremely large delay values
- Handling of wrong number of output values from commands
- Primary node delay correction (always 0ms)
- Output truncation detection and warnings
- Timeout behavior with both short and long timeout values
Test Files:
- test.sh: Main regression test with 7 comprehensive test scenarios
- test_validation.sh: Validation and edge case testing with 6 test scenarios
- test_parsing.sh: Unit test for parsing logic and output format validation
- README: Complete documentation of test coverage and expected behavior
Test Improvements:
- Intelligent wait loops replacing fixed sleeps for better reliability
- Proper error detection and reporting mechanisms
- Comprehensive log analysis and validation
- Better progress reporting during test execution
- Deterministic timing to reduce test flakiness
The test suite ensures the external command feature integrates properly with
existing pgpool functionality and handles various edge cases gracefully.
Tests follow existing pgpool regression test patterns and will be
automatically discovered by the test runner.
Expected Behavior:
- External commands should be executed as configured
- Delay values should be parsed correctly (both int and float)
- Threshold comparisons should work properly with external delays
- Error conditions should be handled gracefully with proper fallbacks
- Commands should timeout appropriately based on configuration
- Timeout errors should provide helpful messages and hints
---
.../083.external_replication_delay/README | 43 +++
.../083.external_replication_delay/test.sh | 304 ++++++++++++++++++
.../test_parsing.sh | 54 ++++
.../test_validation.sh | 285 ++++++++++++++++
4 files changed, 686 insertions(+)
create mode 100644 src/test/regression/tests/083.external_replication_delay/README
create mode 100755 src/test/regression/tests/083.external_replication_delay/test.sh
create mode 100755 src/test/regression/tests/083.external_replication_delay/test_parsing.sh
create mode 100755 src/test/regression/tests/083.external_replication_delay/test_validation.sh
diff --git a/src/test/regression/tests/083.external_replication_delay/README b/src/test/regression/tests/083.external_replication_delay/README
new file mode 100644
index 000000000..1808ba854
--- /dev/null
+++ b/src/test/regression/tests/083.external_replication_delay/README
@@ -0,0 +1,43 @@
+External Replication Delay Command Test
+========================================
+
+This test verifies the external command replication delay source feature.
+
+Test Coverage:
+- Basic external command execution with integer millisecond values
+- Floating-point millisecond value parsing
+- Delay threshold functionality with external commands
+- User switching with sr_check_user parameter
+- Error handling for missing/invalid commands
+- Command execution failure scenarios
+- Command timeout handling with configurable timeout values
+- Input validation for invalid, negative, and extremely large delay values
+- Handling of wrong number of output values
+- Primary node delay correction
+- Output truncation detection
+- Timeout behavior with both short and long timeout values
+
+Files:
+- test.sh: Main test script
+- test_parsing.sh: Unit test for parsing logic
+- test_validation.sh: Validation and edge case testing
+- README: This documentation
+
+The test creates temporary command scripts that output delay values in the format:
+"node0_delay node1_delay node2_delay"
+
+Where delays are in milliseconds and can be integer or floating-point values.
+
+Test Environment:
+- Uses streaming replication mode with 3 nodes
+- Configures sr_check_period = 1 second for faster testing
+- Tests various delay scenarios and threshold behaviors
+
+Expected Behavior:
+- External commands should be executed as configured
+- Delay values should be parsed correctly (both int and float)
+- Threshold comparisons should work properly
+- Error conditions should be handled gracefully
+- Commands should timeout appropriately based on configuration
+- Timeout errors should provide helpful messages and hints
+- Tests should be reliable with proper wait mechanisms instead of fixed sleeps
\ No newline at end of file
diff --git a/src/test/regression/tests/083.external_replication_delay/test.sh b/src/test/regression/tests/083.external_replication_delay/test.sh
new file mode 100755
index 000000000..044ef341e
--- /dev/null
+++ b/src/test/regression/tests/083.external_replication_delay/test.sh
@@ -0,0 +1,304 @@
+#!/usr/bin/env bash
+#-------------------------------------------------------------------
+# test script for external command replication delay source
+#
+source $TESTLIBS
+TESTDIR=testdir
+PG_CTL=$PGBIN/pg_ctl
+PSQL="$PGBIN/psql -X "
+
+rm -fr $TESTDIR
+mkdir $TESTDIR
+cd $TESTDIR
+
+# create test environment
+echo -n "creating test environment..."
+$PGPOOL_SETUP -m s -n 3 || exit 1
+echo "done."
+source ./bashrc.ports
+export PGPORT=$PGPOOL_PORT
+
+# Create external command scripts for testing
+cat > delay_cmd_static.sh << 'EOF'
+#!/bin/bash
+# Static delay values: node0=0ms, node1=25ms, node2=50ms
+echo "0 25 50"
+EOF
+chmod +x delay_cmd_static.sh
+
+cat > delay_cmd_float.sh << 'EOF'
+#!/bin/bash
+# Float delay values: node0=0ms, node1=25.5ms, node2=100.75ms
+echo "0 25.5 100.75"
+EOF
+chmod +x delay_cmd_float.sh
+
+cat > delay_cmd_high.sh << 'EOF'
+#!/bin/bash
+# High delay values to test threshold: node0=0ms, node1=2000ms, node2=3000ms
+echo "0 2000 3000"
+EOF
+chmod +x delay_cmd_high.sh
+
+# ----------------------------------------------------------------------------------------
+echo === Test1: Basic external command with integer millisecond values ===
+# ----------------------------------------------------------------------------------------
+echo "replication_delay_source = 'cmd'" >> etc/pgpool.conf
+echo "replication_delay_source_cmd = './delay_cmd_static.sh'" >> etc/pgpool.conf
+echo "sr_check_period = 1" >> etc/pgpool.conf
+echo "log_standby_delay = 'always'" >> etc/pgpool.conf
+echo "log_min_messages = 'DEBUG1'" >> etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+$PSQL test <<EOF
+CREATE TABLE t1(i INTEGER);
+EOF
+
+# Wait for sr_check to run and populate delay values
+# sr_check_period is 1 second, so wait a bit longer to ensure it runs
+echo "Waiting for sr_check to run..."
+for i in {1..10}; do
+ if grep -q "executing replication delay command" log/pgpool.log 2>/dev/null; then
+ echo "Command executed after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+$PSQL test <<EOF
+SHOW POOL_NODES;
+EOF
+
+# Check that delay values are populated in the log
+grep "executing replication delay command" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: external command was not executed
+ echo "Log contents:"
+ tail -20 log/pgpool.log
+ ./shutdownall
+ exit 1
+fi
+
+# Verify actual delay values were parsed
+if ! $PSQL -t -c "SHOW POOL_NODES" test | grep -E "[0-9]+\.[0-9]+" >/dev/null; then
+ echo "Warning: No delay values found in POOL_NODES output"
+fi
+
+# Check for delay log messages
+grep "Replication of node.*external command" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: external command delay logging not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: basic external command test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test2: External command with floating-point millisecond values ===
+# ----------------------------------------------------------------------------------------
+# Update configuration to use float command
+sed -i.bak "s|delay_cmd_static.sh|delay_cmd_float.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run with float values
+echo "Waiting for sr_check with float values..."
+for i in {1..10}; do
+ if grep -q "executing replication delay command.*delay_cmd_float.sh" log/pgpool.log 2>/dev/null; then
+ echo "Float command executed after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+$PSQL test <<EOF
+SHOW POOL_NODES;
+EOF
+
+# Check that float values are handled correctly
+grep "executing replication delay command.*delay_cmd_float.sh" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: float command was not executed
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: floating-point values test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test3: External command with delay threshold ===
+# ----------------------------------------------------------------------------------------
+# Update configuration to use high delay command and set threshold
+sed -i.bak "s|delay_cmd_float.sh|delay_cmd_high.sh|" etc/pgpool.conf
+echo "delay_threshold_by_time = 1000" >> etc/pgpool.conf
+echo "backend_weight0 = 0" >> etc/pgpool.conf # Force queries to standby normally
+echo "backend_weight2 = 0" >> etc/pgpool.conf # Only use node 1 as standby
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run and detect high delays
+echo "Waiting for sr_check with high delay values..."
+for i in {1..10}; do
+ if grep -q "executing replication delay command.*delay_cmd_high.sh" log/pgpool.log 2>/dev/null; then
+ echo "High delay command executed after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+$PSQL test <<EOF
+SELECT * FROM t1 LIMIT 1;
+EOF
+
+# With high delays (2000ms > 1000ms threshold), query should go to primary (node 0)
+grep "SELECT \* FROM t1 LIMIT 1.*DB node id: 0" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: query was not sent to primary node despite high delay
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: delay threshold test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test4: External command with sr_check_user ===
+# ----------------------------------------------------------------------------------------
+# Test running command as specific user (using current user for test)
+CURRENT_USER=$(whoami)
+echo "sr_check_user = '$CURRENT_USER'" >> etc/pgpool.conf
+sed -i.bak "s|delay_cmd_high.sh|delay_cmd_static.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run with user switching
+echo "Waiting for sr_check with user switching..."
+for i in {1..10}; do
+ if grep -q "executing replication delay command.*su.*$CURRENT_USER" log/pgpool.log 2>/dev/null; then
+ echo "User switching command executed after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check that command was executed with su
+grep "executing replication delay command.*su.*$CURRENT_USER.*delay_cmd_static.sh" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: command was not executed with sr_check_user
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: sr_check_user test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test5: Error handling - missing command ===
+# ----------------------------------------------------------------------------------------
+# Test error handling when command is not configured
+sed -i.bak "s|replication_delay_source_cmd = './delay_cmd_static.sh'|replication_delay_source_cmd = ''|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run with missing command
+echo "Waiting for sr_check with missing command..."
+for i in {1..5}; do
+ if grep -q "replication_delay_source_cmd is not configured" log/pgpool.log 2>/dev/null; then
+ echo "Missing command error detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for error message about missing command
+grep "replication_delay_source_cmd is not configured" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: missing command error not detected
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: error handling test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test6: Error handling - command execution failure ===
+# ----------------------------------------------------------------------------------------
+# Test error handling when command fails
+echo "replication_delay_source_cmd = './nonexistent_command.sh'" >> etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run with failing command
+echo "Waiting for sr_check with failing command..."
+for i in {1..5}; do
+ if grep -q "failed to execute replication delay command" log/pgpool.log 2>/dev/null; then
+ echo "Command failure detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for error message about command execution failure
+grep "failed to execute replication delay command" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: command execution failure not detected
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: command failure test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test7: Command timeout handling ===
+# ----------------------------------------------------------------------------------------
+# Create a command that takes longer than the timeout
+cat > delay_cmd_slow.sh << 'EOF'
+#!/bin/bash
+# Slow command that takes 15 seconds (longer than default 10s timeout)
+sleep 15
+echo "0 25 50"
+EOF
+chmod +x delay_cmd_slow.sh
+
+# Set a short timeout and use the slow command
+sed -i.bak "s|replication_delay_source_cmd = './nonexistent_command.sh'|replication_delay_source_cmd = './delay_cmd_slow.sh'|" etc/pgpool.conf
+echo "replication_delay_source_timeout = 3" >> etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run and timeout
+echo "Waiting for command timeout..."
+for i in {1..15}; do
+ if grep -q "replication delay command timed out" log/pgpool.log 2>/dev/null; then
+ echo "Command timeout detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for timeout error message
+grep "replication delay command timed out after 3 seconds" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: command timeout not detected
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: command timeout test succeeded
+./shutdownall
+
+echo "All external replication delay tests passed!"
+exit 0
\ No newline at end of file
diff --git a/src/test/regression/tests/083.external_replication_delay/test_parsing.sh b/src/test/regression/tests/083.external_replication_delay/test_parsing.sh
new file mode 100755
index 000000000..143da337e
--- /dev/null
+++ b/src/test/regression/tests/083.external_replication_delay/test_parsing.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+#-------------------------------------------------------------------
+# Unit test for external command parsing logic
+# This tests the parsing without needing a full pgpool setup
+#
+
+echo "=== Testing external command output parsing ==="
+
+# Test 1: Integer values
+echo "Test 1: Integer millisecond values"
+echo "0 25 50" > test_output.txt
+echo "Expected: 0ms, 25ms, 50ms"
+echo "Output: $(cat test_output.txt)"
+echo ""
+
+# Test 2: Float values
+echo "Test 2: Floating-point millisecond values"
+echo "0 25.5 100.75" > test_output_float.txt
+echo "Expected: 0ms, 25.5ms, 100.75ms"
+echo "Output: $(cat test_output_float.txt)"
+echo ""
+
+# Test 3: High precision float values
+echo "Test 3: High precision values"
+echo "0 0.001 999.999" > test_output_precision.txt
+echo "Expected: 0ms, 0.001ms, 999.999ms"
+echo "Output: $(cat test_output_precision.txt)"
+echo ""
+
+# Test 4: Edge case - zero values
+echo "Test 4: All zero values"
+echo "0 0 0" > test_output_zeros.txt
+echo "Expected: 0ms, 0ms, 0ms"
+echo "Output: $(cat test_output_zeros.txt)"
+echo ""
+
+# Test 5: Edge case - large values
+echo "Test 5: Large delay values"
+echo "0 5000 10000" > test_output_large.txt
+echo "Expected: 0ms, 5000ms, 10000ms"
+echo "Output: $(cat test_output_large.txt)"
+echo ""
+
+# Test 6: Mixed integer and float
+echo "Test 6: Mixed integer and float values"
+echo "0 25 50.5" > test_output_mixed.txt
+echo "Expected: 0ms, 25ms, 50.5ms"
+echo "Output: $(cat test_output_mixed.txt)"
+echo ""
+
+# Cleanup
+rm -f test_output*.txt
+
+echo "All parsing tests completed. These outputs should be parseable by the external command feature."
\ No newline at end of file
diff --git a/src/test/regression/tests/083.external_replication_delay/test_validation.sh b/src/test/regression/tests/083.external_replication_delay/test_validation.sh
new file mode 100755
index 000000000..4be884c4e
--- /dev/null
+++ b/src/test/regression/tests/083.external_replication_delay/test_validation.sh
@@ -0,0 +1,285 @@
+#!/usr/bin/env bash
+#-------------------------------------------------------------------
+# test script for external command validation and edge cases
+#
+source $TESTLIBS
+TESTDIR=testdir_validation
+PG_CTL=$PGBIN/pg_ctl
+PSQL="$PGBIN/psql -X "
+
+rm -fr $TESTDIR
+mkdir $TESTDIR
+cd $TESTDIR
+
+# create test environment
+echo -n "creating test environment..."
+$PGPOOL_SETUP -m s -n 3 || exit 1
+echo "done."
+source ./bashrc.ports
+export PGPORT=$PGPOOL_PORT
+
+# Create test command scripts
+cat > delay_cmd_validation.sh << 'EOF'
+#!/bin/bash
+# Test validation: output with invalid values
+echo "0 invalid_value 50.5"
+EOF
+chmod +x delay_cmd_validation.sh
+
+cat > delay_cmd_negative.sh << 'EOF'
+#!/bin/bash
+# Test negative values
+echo "0 -25 50"
+EOF
+chmod +x delay_cmd_negative.sh
+
+cat > delay_cmd_large.sh << 'EOF'
+#!/bin/bash
+# Test extremely large values
+echo "0 9999999 50"
+EOF
+chmod +x delay_cmd_large.sh
+
+cat > delay_cmd_wrong_count.sh << 'EOF'
+#!/bin/bash
+# Test wrong number of values (only 2 instead of 3)
+echo "0 25"
+EOF
+chmod +x delay_cmd_wrong_count.sh
+
+cat > delay_cmd_truncated.sh << 'EOF'
+#!/bin/bash
+# Test output that might be truncated (very long line)
+printf "0 25 "
+for i in {1..1000}; do printf "very_long_output_"; done
+echo "50"
+EOF
+chmod +x delay_cmd_truncated.sh
+
+# ----------------------------------------------------------------------------------------
+echo === Test1: Validation of invalid delay values ===
+# ----------------------------------------------------------------------------------------
+echo "replication_delay_source = 'cmd'" >> etc/pgpool.conf
+echo "replication_delay_source_cmd = './delay_cmd_validation.sh'" >> etc/pgpool.conf
+echo "sr_check_period = 1" >> etc/pgpool.conf
+echo "log_standby_delay = 'always'" >> etc/pgpool.conf
+echo "log_min_messages = 'DEBUG1'" >> etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+$PSQL test <<EOF
+CREATE TABLE t1(i INTEGER);
+EOF
+
+# Wait for sr_check to run
+echo "Waiting for validation test..."
+for i in {1..10}; do
+ if grep -q "invalid delay value" log/pgpool.log 2>/dev/null; then
+ echo "Validation error detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for validation warning
+grep "invalid delay value 'invalid_value' for node" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: validation warning not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: invalid value validation test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test2: Negative delay values ===
+# ----------------------------------------------------------------------------------------
+sed -i.bak "s|delay_cmd_validation.sh|delay_cmd_negative.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run
+echo "Waiting for negative value test..."
+for i in {1..10}; do
+ if grep -q "negative delay value" log/pgpool.log 2>/dev/null; then
+ echo "Negative value warning detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for negative value warning
+grep "negative delay value.*for node" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: negative value warning not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: negative value validation test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test3: Extremely large delay values ===
+# ----------------------------------------------------------------------------------------
+sed -i.bak "s|delay_cmd_negative.sh|delay_cmd_large.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run
+echo "Waiting for large value test..."
+for i in {1..10}; do
+ if grep -q "extremely large delay value" log/pgpool.log 2>/dev/null; then
+ echo "Large value warning detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for large value warning
+grep "extremely large delay value.*for node" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: large value warning not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: large value validation test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test4: Wrong number of output values ===
+# ----------------------------------------------------------------------------------------
+sed -i.bak "s|delay_cmd_large.sh|delay_cmd_wrong_count.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run
+echo "Waiting for wrong count test..."
+for i in {1..10}; do
+ if grep -q "returned.*values, expected" log/pgpool.log 2>/dev/null; then
+ echo "Wrong count warning detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for wrong count warning
+grep "returned.*values, expected.*Command should output one delay value per backend node" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: wrong count warning not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: wrong count validation test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test5: Primary node non-zero delay handling ===
+# ----------------------------------------------------------------------------------------
+cat > delay_cmd_primary_nonzero.sh << 'EOF'
+#!/bin/bash
+# Test primary node with non-zero delay (should be corrected to 0)
+echo "100 25 50"
+EOF
+chmod +x delay_cmd_primary_nonzero.sh
+
+sed -i.bak "s|delay_cmd_wrong_count.sh|delay_cmd_primary_nonzero.sh|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for sr_check to run
+echo "Waiting for primary non-zero delay test..."
+for i in {1..10}; do
+ if grep -q "primary node.*reported non-zero delay" log/pgpool.log 2>/dev/null; then
+ echo "Primary non-zero delay detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for primary node correction
+grep "primary node.*reported non-zero delay.*setting to 0" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: primary node delay correction not found
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: primary node delay correction test succeeded
+./shutdownall
+
+# ----------------------------------------------------------------------------------------
+echo === Test6: Command timeout with different timeout values ===
+# ----------------------------------------------------------------------------------------
+cat > delay_cmd_timeout.sh << 'EOF'
+#!/bin/bash
+# Command that takes 5 seconds
+sleep 5
+echo "0 25 50"
+EOF
+chmod +x delay_cmd_timeout.sh
+
+# Test with timeout shorter than command duration
+sed -i.bak "s|delay_cmd_primary_nonzero.sh|delay_cmd_timeout.sh|" etc/pgpool.conf
+echo "replication_delay_source_timeout = 2" >> etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for timeout
+echo "Waiting for timeout test (2s timeout, 5s command)..."
+for i in {1..10}; do
+ if grep -q "replication delay command timed out after 2 seconds" log/pgpool.log 2>/dev/null; then
+ echo "Timeout detected after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Check for timeout message
+grep "replication delay command timed out after 2 seconds" log/pgpool.log >/dev/null 2>&1
+if [ $? != 0 ];then
+ echo fail: timeout not detected
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: timeout test succeeded
+./shutdownall
+
+# Test with timeout longer than command duration
+sed -i.bak "s|replication_delay_source_timeout = 2|replication_delay_source_timeout = 10|" etc/pgpool.conf
+
+./startall
+wait_for_pgpool_startup
+
+# Wait for successful execution
+echo "Waiting for successful execution (10s timeout, 5s command)..."
+for i in {1..15}; do
+ if grep -q "executing replication delay command.*delay_cmd_timeout.sh" log/pgpool.log 2>/dev/null; then
+ echo "Command executed successfully after ${i} seconds"
+ break
+ fi
+ sleep 1
+done
+
+# Should not timeout this time
+if grep -q "replication delay command timed out" log/pgpool.log 2>/dev/null; then
+ echo fail: command should not have timed out with 10s timeout
+ ./shutdownall
+ exit 1
+fi
+
+echo ok: extended timeout test succeeded
+./shutdownall
+
+echo "All validation tests passed!"
+exit 0
\ No newline at end of file
--
2.51.0
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Proposal: recent access based routing for primary-replica setups
In-Reply-To: <CACeKOO000N8gUodkXt_1V=XQYx_utxXdyOxkzV7GYSPpdHnTfg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox