agora inbox for [email protected]
help / color / mirror / Atom feedRE: pgbench - doCustom cleanup
25+ messages / 7 participants
[nested] [flat]
* RE: pgbench - doCustom cleanup
@ 2018-11-17 10:46 Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Fabien COELHO @ 2018-11-17 10:46 UTC (permalink / raw)
To: Jamison, Kirk <[email protected]>; +Cc: PostgreSQL Developers <[email protected]>
> Attached is a v3, where I have updated inaccurate comments.
Attached v4 is a rebase after 409231919443984635b7ae9b7e2e261ab984eb1e
--
Fabien.
Attachments:
[text/x-diff] pgbench-state-change-4.patch (17.5K, ../../alpine.DEB.2.21.1811171144230.16105@lancre/2-pgbench-state-change-4.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 73d3de0677..ed6ff75426 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -295,23 +295,31 @@ typedef enum
/*
* The client must first choose a script to execute. Once chosen, it can
* either be throttled (state CSTATE_START_THROTTLE under --rate) or start
- * right away (state CSTATE_START_TX).
+ * right away (state CSTATE_START_TX) or not start at all if the timer was
+ * exceeded (state CSTATE_FINISHED).
*/
CSTATE_CHOOSE_SCRIPT,
/*
* In CSTATE_START_THROTTLE state, we calculate when to begin the next
* transaction, and advance to CSTATE_THROTTLE. CSTATE_THROTTLE state
- * sleeps until that moment. (If throttling is not enabled, doCustom()
- * falls directly through from CSTATE_START_THROTTLE to CSTATE_START_TX.)
+ * sleeps until that moment.
+ *
+ * It may also detect that the next transaction would start beyond the end
+ * of run, and switch to CSTATE_FINISHED.
*/
CSTATE_START_THROTTLE,
CSTATE_THROTTLE,
/*
* CSTATE_START_TX performs start-of-transaction processing. Establishes
- * a new connection for the transaction, in --connect mode, and records
- * the transaction start time.
+ * a new connection for the transaction in --connect mode, records
+ * the transaction start time, and proceed to the first command.
+ *
+ * Note: once a script is started, it will either error or run till
+ * its end, where it may be interrupted. It is not interrupted while
+ * running, so pgbench --time is to be understood as tx are allowed to
+ * start in that time, and will finish when their work is completed.
*/
CSTATE_START_TX,
@@ -324,9 +332,6 @@ typedef enum
* and we enter the CSTATE_SLEEP state to wait for it to expire. Other
* meta-commands are executed immediately.
*
- * CSTATE_SKIP_COMMAND for conditional branches which are not executed,
- * quickly skip commands that do not need any evaluation.
- *
* CSTATE_WAIT_RESULT waits until we get a result set back from the server
* for the current command.
*
@@ -334,19 +339,25 @@ typedef enum
*
* CSTATE_END_COMMAND records the end-of-command timestamp, increments the
* command counter, and loops back to CSTATE_START_COMMAND state.
+ *
+ * CSTATE_SKIP_COMMAND is used by conditional branches which are not
+ * executed. It quickly skip commands that do not need any evaluation.
+ * This state can move forward several commands, till there is something
+ * to do or the end of the script.
*/
CSTATE_START_COMMAND,
- CSTATE_SKIP_COMMAND,
CSTATE_WAIT_RESULT,
CSTATE_SLEEP,
CSTATE_END_COMMAND,
+ CSTATE_SKIP_COMMAND,
/*
- * CSTATE_END_TX performs end-of-transaction processing. Calculates
- * latency, and logs the transaction. In --connect mode, closes the
- * current connection. Chooses the next script to execute and starts over
- * in CSTATE_START_THROTTLE state, or enters CSTATE_FINISHED if we have no
- * more work to do.
+ * CSTATE_END_TX performs end-of-transaction processing. It calculates
+ * latency, and logs the transaction. In --connect mode, it closes the
+ * current connection.
+ *
+ * Then either starts over in CSTATE_CHOOSE_SCRIPT, or enters CSTATE_FINISHED
+ * if we have no more work to do.
*/
CSTATE_END_TX,
@@ -2821,16 +2832,13 @@ evaluateSleep(CState *st, int argc, char **argv, int *usecs)
/*
* Advance the state machine of a connection, if possible.
+ *
+ * All state changes are performed within this function called
+ * by threadRun.
*/
static void
doCustom(TState *thread, CState *st, StatsData *agg)
{
- PGresult *res;
- Command *command;
- instr_time now;
- bool end_tx_processed = false;
- int64 wait;
-
/*
* gettimeofday() isn't free, so we get the current timestamp lazily the
* first time it's needed, and reuse the same value throughout this
@@ -2839,37 +2847,44 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* means "not set yet". Reset "now" when we execute shell commands or
* expressions, which might take a non-negligible amount of time, though.
*/
+ instr_time now;
INSTR_TIME_SET_ZERO(now);
/*
* Loop in the state machine, until we have to wait for a result from the
- * server (or have to sleep, for throttling or for \sleep).
+ * server or have to sleep for throttling or \sleep.
*
* Note: In the switch-statement below, 'break' will loop back here,
* meaning "continue in the state machine". Return is used to return to
- * the caller.
+ * the caller, giving the thread opportunity to move forward another client.
*/
for (;;)
{
+ PGresult *res;
+ Command *command;
+
switch (st->state)
{
/*
* Select transaction to run.
*/
case CSTATE_CHOOSE_SCRIPT:
-
st->use_file = chooseScript(thread);
if (debug)
fprintf(stderr, "client %d executing script \"%s\"\n", st->id,
sql_script[st->use_file].desc);
- if (throttle_delay > 0)
+ /* check stack consistency */
+ Assert(conditional_stack_empty(st->cstack));
+
+ if (timer_exceeded)
+ st->state = CSTATE_FINISHED;
+ else if (throttle_delay > 0)
st->state = CSTATE_START_THROTTLE;
else
st->state = CSTATE_START_TX;
- /* check consistency */
- Assert(conditional_stack_empty(st->cstack));
+
break;
/*
@@ -2887,21 +2902,10 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* away.
*/
Assert(throttle_delay > 0);
- wait = getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger += getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
- /*
- * stop client if next transaction is beyond pgbench end of
- * execution
- */
- if (duration > 0 && st->txn_scheduled > end_time)
- {
- st->state = CSTATE_FINISHED;
- break;
- }
-
/*
* If --latency-limit is used, and this slot is already late
* so that the transaction will miss the latency limit even if
@@ -2913,20 +2917,20 @@ doCustom(TState *thread, CState *st, StatsData *agg)
{
int64 now_us;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
now_us = INSTR_TIME_GET_MICROSEC(now);
+
while (thread->throttle_trigger < now_us - latency_limit &&
(nxacts <= 0 || st->cnt < nxacts))
{
processXactStats(thread, st, &now, true, agg);
/* next rendez-vous */
- wait = getPoissonRand(&thread->ts_throttle_rs,
- throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger +=
+ getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
}
- /* stop client if -t exceeded */
+
+ /* stop client if -t was exceeded in the previous skip loop */
if (nxacts > 0 && st->cnt >= nxacts)
{
st->state = CSTATE_FINISHED;
@@ -2934,38 +2938,45 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
}
+ /*
+ * stop client if next transaction is beyond pgbench end of
+ * execution.
+ */
+ if (duration > 0 && st->txn_scheduled > end_time)
+ {
+ st->state = CSTATE_FINISHED;
+ break;
+ }
+
st->state = CSTATE_THROTTLE;
- if (debug)
- fprintf(stderr, "client %d throttling " INT64_FORMAT " us\n",
- st->id, wait);
break;
/*
* Wait until it's time to start next transaction.
*/
case CSTATE_THROTTLE:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
- /* Else done sleeping, start the transaction */
- st->state = CSTATE_START_TX;
+ /* done sleeping, but do not start if transaction if we are done */
+ if (timer_exceeded)
+ st->state = CSTATE_FINISHED;
+ else
+ st->state = CSTATE_START_TX;
break;
/* Start new transaction */
case CSTATE_START_TX:
- /*
- * Establish connection on first call, or if is_connect is
- * true.
- */
+ /* establish connection if needed, i.e. under --connect */
if (st->con == NULL)
{
instr_time start;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
start = now;
if ((st->con = doConnect()) == NULL)
{
@@ -2981,28 +2992,20 @@ doCustom(TState *thread, CState *st, StatsData *agg)
memset(st->prepared, 0, sizeof(st->prepared));
}
+ /* record transaction start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->txn_begin = now;
+
/*
- * Record transaction start time under logging, progress or
- * throttling.
+ * When not throttling, this is also the transaction's
+ * scheduled start time.
*/
- if (use_log || progress || throttle_delay || latency_limit ||
- per_script_stats)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->txn_begin = now;
-
- /*
- * When not throttling, this is also the transaction's
- * scheduled start time.
- */
- if (!throttle_delay)
- st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
- }
+ if (!throttle_delay)
+ st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
/* Begin with the first command */
- st->command = 0;
st->state = CSTATE_START_COMMAND;
+ st->command = 0;
break;
/*
@@ -3021,17 +3024,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
}
- /*
- * Record statement start time if per-command latencies are
- * requested
- */
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->stmt_begin = now;
- }
+ /* record statement start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->stmt_begin = now;
+ /* execute the command */
if (command->type == SQL_COMMAND)
{
if (!sendCommand(st, command))
@@ -3074,8 +3071,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
}
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
st->state = CSTATE_SLEEP;
break;
@@ -3126,10 +3123,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
else /* elif */
{
- /*
- * we should get here only if the "elif"
- * needed evaluation
- */
+ /* we should get here only if the "elif" needed evaluation */
Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
}
@@ -3161,43 +3155,23 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
else if (command->meta == META_SETSHELL)
{
- bool ret = runShellCommand(st, argv[1], argv + 2, argc - 2);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
+ if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
{
commandFailed(st, "setshell", "execution of meta-command failed");
st->state = CSTATE_ABORTED;
break;
}
- else
- {
- /* succeeded */
- }
+ /* else success */
}
else if (command->meta == META_SHELL)
{
- bool ret = runShellCommand(st, NULL, argv + 1, argc - 1);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
+ if (!runShellCommand(st, NULL, argv + 1, argc - 1))
{
commandFailed(st, "shell", "execution of meta-command failed");
st->state = CSTATE_ABORTED;
break;
}
- else
- {
- /* succeeded */
- }
+ /* else success */
}
move_to_end_command:
@@ -3299,6 +3273,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
if (st->state != CSTATE_SKIP_COMMAND)
+ /* out of quick skip command loop */
break;
}
break;
@@ -3348,10 +3323,9 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* instead of CSTATE_START_TX.
*/
case CSTATE_SLEEP:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
if (INSTR_TIME_GET_MICROSEC(now) < st->sleep_until)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
/* Else done sleeping. */
st->state = CSTATE_END_COMMAND;
break;
@@ -3366,17 +3340,13 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* in thread-local data structure, if per-command latencies
* are requested.
*/
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
- /* XXX could use a mutex here, but we choose not to */
- command = sql_script[st->use_file].commands[st->command];
- addToSimpleStats(&command->stats,
- INSTR_TIME_GET_DOUBLE(now) -
- INSTR_TIME_GET_DOUBLE(st->stmt_begin));
- }
+ /* XXX could use a mutex here, but we choose not to */
+ command = sql_script[st->use_file].commands[st->command];
+ addToSimpleStats(&command->stats,
+ INSTR_TIME_GET_DOUBLE(now) -
+ INSTR_TIME_GET_DOUBLE(st->stmt_begin));
/* Go ahead with next command, to be executed or skipped */
st->command++;
@@ -3385,19 +3355,15 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
/*
- * End of transaction.
+ * End of transaction (end of script, really).
*/
case CSTATE_END_TX:
/* transaction finished: calculate latency and do log */
processXactStats(thread, st, &now, false, agg);
- /* conditional stack must be empty */
- if (!conditional_stack_empty(st->cstack))
- {
- fprintf(stderr, "end of script reached within a conditional, missing \\endif\n");
- exit(1);
- }
+ /* missing \endif... cannot happen if CheckConditional was okay */
+ Assert(conditional_stack_empty(st->cstack));
if (is_connect)
{
@@ -3411,26 +3377,17 @@ doCustom(TState *thread, CState *st, StatsData *agg)
st->state = CSTATE_FINISHED;
break;
}
+ else
+ {
+ /* next transaction */
+ st->state = CSTATE_CHOOSE_SCRIPT;
- /*
- * No transaction is underway anymore.
- */
- st->state = CSTATE_CHOOSE_SCRIPT;
-
- /*
- * If we paced through all commands in the script in this
- * loop, without returning to the caller even once, do it now.
- * This gives the thread a chance to process other
- * connections, and to do progress reporting. This can
- * currently only happen if the script consists entirely of
- * meta-commands.
- */
- if (end_tx_processed)
+ /*
+ * Ensure that we always return on this point, so as
+ * to avoid an infinite loop if the script only contains
+ * meta commands.
+ */
return;
- else
- {
- end_tx_processed = true;
- break;
}
/*
@@ -3544,8 +3501,7 @@ processXactStats(TState *thread, CState *st, instr_time *now,
if (detailed && !skipped)
{
- if (INSTR_TIME_IS_ZERO(*now))
- INSTR_TIME_SET_CURRENT(*now);
+ INSTR_TIME_SET_CURRENT_LAZY(*now);
/* compute latency & lag */
latency = INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled;
@@ -5809,7 +5765,7 @@ threadRun(void *arg)
if (!is_connect)
{
- /* make connections to the database */
+ /* make connections to the database before starting */
for (i = 0; i < nstate; i++)
{
if ((state[i].con = doConnect()) == NULL)
@@ -5845,14 +5801,7 @@ threadRun(void *arg)
{
CState *st = &state[i];
- if (st->state == CSTATE_THROTTLE && timer_exceeded)
- {
- /* interrupt client that has not started a transaction */
- st->state = CSTATE_FINISHED;
- finishCon(st);
- remains--;
- }
- else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
+ if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
{
/* a nap from the script, or under throttling */
int64 this_usec;
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index f968444671..c46f6825bb 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -20,6 +20,8 @@
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
*
+ * INSTR_TIME_SET_CURRENT_LAZY(t) set t to current time if t is zero
+ *
* INSTR_TIME_ADD(x, y) x += y
*
* INSTR_TIME_SUBTRACT(x, y) x -= y
@@ -245,4 +247,9 @@ GetTimerFrequency(void)
#endif /* WIN32 */
+/* same macro on all platforms */
+#define INSTR_TIME_SET_CURRENT_LAZY(t) \
+ if (INSTR_TIME_IS_ZERO(t)) \
+ INSTR_TIME_SET_CURRENT(t)
+
#endif /* INSTR_TIME_H */
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
@ 2018-11-19 23:02 ` Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-19 23:02 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
On 2018-Nov-17, Fabien COELHO wrote:
>
> > Attached is a v3, where I have updated inaccurate comments.
>
> Attached v4 is a rebase after 409231919443984635b7ae9b7e2e261ab984eb1e
Attached v5. I thought that separating the part that executes the
command was an obvious readability improvement. Tests still pass,
though maybe I broke something inadvertently. (I started by thinking
"does this block require a 'fall-through' comment?" The 600 line
function is pretty hard to read with the multiple messy switches and
conditionals; split into 400 + 200 subroutine it's nicer to reason
about.)
Do we really use the word "move" to talk about state changes? It sounds
very odd to me. I would change that to "transition" -- would anybody
object to that? (Not changed in v5.)
On INSTR_TIME_SET_CURRENT_LAZY(), you cannot just put an "if" inside a
macro -- consider this:
if (foo)
INSTR_TIME_SET_CURRENT_LAZY(bar);
else
something_else();
Which "if" is the else now attached to? Now maybe the C standard has an
answer for that (I don't know what it is), but it's hard to read and
likely the compiler will complain anyway. I wrapped it in "do { }
while(0)" as is customary.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
[text/x-diff] pgbench-state-change-5.patch (25.6K, ../../[email protected]/2-pgbench-state-change-5.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 73d3de0677..df14af5259 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -294,24 +294,32 @@ typedef enum
{
/*
* The client must first choose a script to execute. Once chosen, it can
- * either be throttled (state CSTATE_START_THROTTLE under --rate) or start
- * right away (state CSTATE_START_TX).
+ * either be throttled (state CSTATE_START_THROTTLE under --rate), start
+ * right away (state CSTATE_START_TX) or not start at all if the timer was
+ * exceeded (state CSTATE_FINISHED).
*/
CSTATE_CHOOSE_SCRIPT,
/*
* In CSTATE_START_THROTTLE state, we calculate when to begin the next
* transaction, and advance to CSTATE_THROTTLE. CSTATE_THROTTLE state
- * sleeps until that moment. (If throttling is not enabled, doCustom()
- * falls directly through from CSTATE_START_THROTTLE to CSTATE_START_TX.)
+ * sleeps until that moment.
+ *
+ * It may also detect that the next transaction would start beyond the end
+ * of run, and switch to CSTATE_FINISHED.
*/
CSTATE_START_THROTTLE,
CSTATE_THROTTLE,
/*
* CSTATE_START_TX performs start-of-transaction processing. Establishes
- * a new connection for the transaction, in --connect mode, and records
- * the transaction start time.
+ * a new connection for the transaction in --connect mode, records
+ * the transaction start time, and proceed to the first command.
+ *
+ * Note: once a script is started, it will either error or run till
+ * its end, where it may be interrupted. It is not interrupted while
+ * running, so pgbench --time is to be understood as tx are allowed to
+ * start in that time, and will finish when their work is completed.
*/
CSTATE_START_TX,
@@ -324,9 +332,6 @@ typedef enum
* and we enter the CSTATE_SLEEP state to wait for it to expire. Other
* meta-commands are executed immediately.
*
- * CSTATE_SKIP_COMMAND for conditional branches which are not executed,
- * quickly skip commands that do not need any evaluation.
- *
* CSTATE_WAIT_RESULT waits until we get a result set back from the server
* for the current command.
*
@@ -334,19 +339,25 @@ typedef enum
*
* CSTATE_END_COMMAND records the end-of-command timestamp, increments the
* command counter, and loops back to CSTATE_START_COMMAND state.
+ *
+ * CSTATE_SKIP_COMMAND is used by conditional branches which are not
+ * executed. It quickly skip commands that do not need any evaluation.
+ * This state can move forward several commands, till there is something
+ * to do or the end of the script.
*/
CSTATE_START_COMMAND,
- CSTATE_SKIP_COMMAND,
CSTATE_WAIT_RESULT,
CSTATE_SLEEP,
CSTATE_END_COMMAND,
+ CSTATE_SKIP_COMMAND,
/*
- * CSTATE_END_TX performs end-of-transaction processing. Calculates
- * latency, and logs the transaction. In --connect mode, closes the
- * current connection. Chooses the next script to execute and starts over
- * in CSTATE_START_THROTTLE state, or enters CSTATE_FINISHED if we have no
- * more work to do.
+ * CSTATE_END_TX performs end-of-transaction processing. It calculates
+ * latency, and logs the transaction. In --connect mode, it closes the
+ * current connection.
+ *
+ * Then either starts over in CSTATE_CHOOSE_SCRIPT, or enters CSTATE_FINISHED
+ * if we have no more work to do.
*/
CSTATE_END_TX,
@@ -575,6 +586,7 @@ static void processXactStats(TState *thread, CState *st, instr_time *now,
static void pgbench_error(const char *fmt,...) pg_attribute_printf(1, 2);
static void addScript(ParsedScript script);
static void *threadRun(void *arg);
+static instr_time do_execute_command(TState *thread, CState *st, instr_time now);
static void finishCon(CState *st);
static void setalarm(int seconds);
static socket_set *alloc_socket_set(int count);
@@ -2821,16 +2833,12 @@ evaluateSleep(CState *st, int argc, char **argv, int *usecs)
/*
* Advance the state machine of a connection, if possible.
+ *
+ * All state changes are performed within this function called by threadRun.
*/
static void
doCustom(TState *thread, CState *st, StatsData *agg)
{
- PGresult *res;
- Command *command;
- instr_time now;
- bool end_tx_processed = false;
- int64 wait;
-
/*
* gettimeofday() isn't free, so we get the current timestamp lazily the
* first time it's needed, and reuse the same value throughout this
@@ -2839,37 +2847,45 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* means "not set yet". Reset "now" when we execute shell commands or
* expressions, which might take a non-negligible amount of time, though.
*/
+ instr_time now;
+
INSTR_TIME_SET_ZERO(now);
/*
* Loop in the state machine, until we have to wait for a result from the
- * server (or have to sleep, for throttling or for \sleep).
+ * server or have to sleep for throttling or \sleep.
*
* Note: In the switch-statement below, 'break' will loop back here,
* meaning "continue in the state machine". Return is used to return to
- * the caller.
+ * the caller, giving the thread the opportunity to advance another client.
*/
for (;;)
{
+ PGresult *res;
+ Command *command;
+
switch (st->state)
{
/*
* Select transaction to run.
*/
case CSTATE_CHOOSE_SCRIPT:
-
st->use_file = chooseScript(thread);
if (debug)
fprintf(stderr, "client %d executing script \"%s\"\n", st->id,
sql_script[st->use_file].desc);
- if (throttle_delay > 0)
+ /* check stack consistency */
+ Assert(conditional_stack_empty(st->cstack));
+
+ if (timer_exceeded)
+ st->state = CSTATE_FINISHED;
+ else if (throttle_delay > 0)
st->state = CSTATE_START_THROTTLE;
else
st->state = CSTATE_START_TX;
- /* check consistency */
- Assert(conditional_stack_empty(st->cstack));
+
break;
/*
@@ -2887,22 +2903,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* away.
*/
Assert(throttle_delay > 0);
- wait = getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger += getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
/*
- * stop client if next transaction is beyond pgbench end of
- * execution
- */
- if (duration > 0 && st->txn_scheduled > end_time)
- {
- st->state = CSTATE_FINISHED;
- break;
- }
-
- /*
* If --latency-limit is used, and this slot is already late
* so that the transaction will miss the latency limit even if
* it completed immediately, we skip this time slot and
@@ -2913,20 +2918,20 @@ doCustom(TState *thread, CState *st, StatsData *agg)
{
int64 now_us;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
now_us = INSTR_TIME_GET_MICROSEC(now);
+
while (thread->throttle_trigger < now_us - latency_limit &&
(nxacts <= 0 || st->cnt < nxacts))
{
processXactStats(thread, st, &now, true, agg);
/* next rendez-vous */
- wait = getPoissonRand(&thread->ts_throttle_rs,
- throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger +=
+ getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
}
- /* stop client if -t exceeded */
+
+ /* stop client if -t was exceeded in the previous skip loop */
if (nxacts > 0 && st->cnt >= nxacts)
{
st->state = CSTATE_FINISHED;
@@ -2934,38 +2939,42 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
}
+ /*
+ * stop client if next transaction is beyond pgbench end of
+ * execution.
+ */
+ if (duration > 0 && st->txn_scheduled > end_time)
+ {
+ st->state = CSTATE_FINISHED;
+ break;
+ }
+
st->state = CSTATE_THROTTLE;
- if (debug)
- fprintf(stderr, "client %d throttling " INT64_FORMAT " us\n",
- st->id, wait);
break;
/*
* Wait until it's time to start next transaction.
*/
case CSTATE_THROTTLE:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
- return; /* Still sleeping, nothing to do here */
- /* Else done sleeping, start the transaction */
- st->state = CSTATE_START_TX;
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
+ if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
+ return; /* still sleeping, nothing to do here */
+
+ /* done sleeping, but do not start transaction if we are done */
+ st->state = timer_exceeded ? CSTATE_FINISHED : CSTATE_START_TX;
break;
/* Start new transaction */
case CSTATE_START_TX:
- /*
- * Establish connection on first call, or if is_connect is
- * true.
- */
+ /* establish connection if needed, i.e. under --connect */
if (st->con == NULL)
{
instr_time start;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
start = now;
if ((st->con = doConnect()) == NULL)
{
@@ -2981,235 +2990,28 @@ doCustom(TState *thread, CState *st, StatsData *agg)
memset(st->prepared, 0, sizeof(st->prepared));
}
- /*
- * Record transaction start time under logging, progress or
- * throttling.
- */
- if (use_log || progress || throttle_delay || latency_limit ||
- per_script_stats)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->txn_begin = now;
+ /* record transaction start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->txn_begin = now;
- /*
- * When not throttling, this is also the transaction's
- * scheduled start time.
- */
- if (!throttle_delay)
- st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
- }
+ /*
+ * When not throttling, this is also the transaction's
+ * scheduled start time.
+ */
+ if (!throttle_delay)
+ st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
/* Begin with the first command */
- st->command = 0;
st->state = CSTATE_START_COMMAND;
+ st->command = 0;
break;
/*
* Send a command to server (or execute a meta-command)
*/
case CSTATE_START_COMMAND:
- command = sql_script[st->use_file].commands[st->command];
- /*
- * If we reached the end of the script, move to end-of-xact
- * processing.
- */
- if (command == NULL)
- {
- st->state = CSTATE_END_TX;
- break;
- }
-
- /*
- * Record statement start time if per-command latencies are
- * requested
- */
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->stmt_begin = now;
- }
-
- if (command->type == SQL_COMMAND)
- {
- if (!sendCommand(st, command))
- {
- commandFailed(st, "SQL", "SQL command send failed");
- st->state = CSTATE_ABORTED;
- }
- else
- st->state = CSTATE_WAIT_RESULT;
- }
- else if (command->type == META_COMMAND)
- {
- int argc = command->argc,
- i;
- char **argv = command->argv;
-
- if (debug)
- {
- fprintf(stderr, "client %d executing \\%s", st->id, argv[0]);
- for (i = 1; i < argc; i++)
- fprintf(stderr, " %s", argv[i]);
- fprintf(stderr, "\n");
- }
-
- if (command->meta == META_SLEEP)
- {
- /*
- * A \sleep doesn't execute anything, we just get the
- * delay from the argument, and enter the CSTATE_SLEEP
- * state. (The per-command latency will be recorded
- * in CSTATE_SLEEP state, not here, after the delay
- * has elapsed.)
- */
- int usec;
-
- if (!evaluateSleep(st, argc, argv, &usec))
- {
- commandFailed(st, "sleep", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
-
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
- st->state = CSTATE_SLEEP;
- break;
- }
- else if (command->meta == META_SET ||
- command->meta == META_IF ||
- command->meta == META_ELIF)
- {
- /* backslash commands with an expression to evaluate */
- PgBenchExpr *expr = command->expr;
- PgBenchValue result;
-
- if (command->meta == META_ELIF &&
- conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
- {
- /*
- * elif after executed block, skip eval and wait
- * for endif
- */
- conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
- goto move_to_end_command;
- }
-
- if (!evaluateExpr(thread, st, expr, &result))
- {
- commandFailed(st, argv[0], "evaluation of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
-
- if (command->meta == META_SET)
- {
- if (!putVariableValue(st, argv[0], argv[1], &result))
- {
- commandFailed(st, "set", "assignment of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- }
- else /* if and elif evaluated cases */
- {
- bool cond = valueTruth(&result);
-
- /* execute or not depending on evaluated condition */
- if (command->meta == META_IF)
- {
- conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- else /* elif */
- {
- /*
- * we should get here only if the "elif"
- * needed evaluation
- */
- Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
- conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- }
- }
- else if (command->meta == META_ELSE)
- {
- switch (conditional_stack_peek(st->cstack))
- {
- case IFSTATE_TRUE:
- conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
- break;
- case IFSTATE_FALSE: /* inconsistent if active */
- case IFSTATE_IGNORED: /* inconsistent if active */
- case IFSTATE_NONE: /* else without if */
- case IFSTATE_ELSE_TRUE: /* else after else */
- case IFSTATE_ELSE_FALSE: /* else after else */
- default:
- /* dead code if conditional check is ok */
- Assert(false);
- }
- goto move_to_end_command;
- }
- else if (command->meta == META_ENDIF)
- {
- Assert(!conditional_stack_empty(st->cstack));
- conditional_stack_pop(st->cstack);
- goto move_to_end_command;
- }
- else if (command->meta == META_SETSHELL)
- {
- bool ret = runShellCommand(st, argv[1], argv + 2, argc - 2);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
- {
- commandFailed(st, "setshell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- else
- {
- /* succeeded */
- }
- }
- else if (command->meta == META_SHELL)
- {
- bool ret = runShellCommand(st, NULL, argv + 1, argc - 1);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
- {
- commandFailed(st, "shell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- else
- {
- /* succeeded */
- }
- }
-
- move_to_end_command:
-
- /*
- * executing the expression or shell command might take a
- * non-negligible amount of time, so reset 'now'
- */
- INSTR_TIME_SET_ZERO(now);
-
- st->state = CSTATE_END_COMMAND;
- }
+ now = do_execute_command(thread, st, now);
break;
/*
@@ -3299,6 +3101,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
if (st->state != CSTATE_SKIP_COMMAND)
+ /* out of quick skip command loop */
break;
}
break;
@@ -3348,10 +3151,9 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* instead of CSTATE_START_TX.
*/
case CSTATE_SLEEP:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
if (INSTR_TIME_GET_MICROSEC(now) < st->sleep_until)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
/* Else done sleeping. */
st->state = CSTATE_END_COMMAND;
break;
@@ -3366,17 +3168,13 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* in thread-local data structure, if per-command latencies
* are requested.
*/
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
- /* XXX could use a mutex here, but we choose not to */
- command = sql_script[st->use_file].commands[st->command];
- addToSimpleStats(&command->stats,
- INSTR_TIME_GET_DOUBLE(now) -
- INSTR_TIME_GET_DOUBLE(st->stmt_begin));
- }
+ /* XXX could use a mutex here, but we choose not to */
+ command = sql_script[st->use_file].commands[st->command];
+ addToSimpleStats(&command->stats,
+ INSTR_TIME_GET_DOUBLE(now) -
+ INSTR_TIME_GET_DOUBLE(st->stmt_begin));
/* Go ahead with next command, to be executed or skipped */
st->command++;
@@ -3385,19 +3183,15 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
/*
- * End of transaction.
+ * End of transaction (end of script, really).
*/
case CSTATE_END_TX:
/* transaction finished: calculate latency and do log */
processXactStats(thread, st, &now, false, agg);
- /* conditional stack must be empty */
- if (!conditional_stack_empty(st->cstack))
- {
- fprintf(stderr, "end of script reached within a conditional, missing \\endif\n");
- exit(1);
- }
+ /* missing \endif... cannot happen if CheckConditional was okay */
+ Assert(conditional_stack_empty(st->cstack));
if (is_connect)
{
@@ -3411,26 +3205,17 @@ doCustom(TState *thread, CState *st, StatsData *agg)
st->state = CSTATE_FINISHED;
break;
}
-
- /*
- * No transaction is underway anymore.
- */
- st->state = CSTATE_CHOOSE_SCRIPT;
-
- /*
- * If we paced through all commands in the script in this
- * loop, without returning to the caller even once, do it now.
- * This gives the thread a chance to process other
- * connections, and to do progress reporting. This can
- * currently only happen if the script consists entirely of
- * meta-commands.
- */
- if (end_tx_processed)
- return;
else
{
- end_tx_processed = true;
- break;
+ /* next transaction */
+ st->state = CSTATE_CHOOSE_SCRIPT;
+
+ /*
+ * Ensure that we always return on this point, so as
+ * to avoid an infinite loop if the script only contains
+ * meta commands.
+ */
+ return;
}
/*
@@ -3445,6 +3230,189 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
/*
+ * do_execute_command -- subroutine for doCustom
+ * Executes the current command and advance the state machine.
+ *
+ * Returns the time that's current at command finish, which can be the input
+ * time if we consider that the time taken to execute would be negligible.
+ */
+static instr_time
+do_execute_command(TState *thread, CState *st, instr_time now)
+{
+ Command *command;
+
+ command = sql_script[st->use_file].commands[st->command];
+
+ /* If we reached the end of the script, move to end-of-xact processing */
+ if (command == NULL)
+ {
+ st->state = CSTATE_END_TX;
+ return now;
+ }
+
+ /* record statement start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->stmt_begin = now;
+
+ /* execute the command */
+ if (command->type == SQL_COMMAND)
+ {
+ if (!sendCommand(st, command))
+ {
+ commandFailed(st, "SQL", "SQL command send failed");
+ st->state = CSTATE_ABORTED;
+ }
+ else
+ st->state = CSTATE_WAIT_RESULT;
+ }
+ else if (command->type == META_COMMAND)
+ {
+ int argc = command->argc;
+ char **argv = command->argv;
+
+ if (debug)
+ {
+ fprintf(stderr, "client %d executing \\%s", st->id, argv[0]);
+ for (int i = 1; i < argc; i++)
+ fprintf(stderr, " %s", argv[i]);
+ fprintf(stderr, "\n");
+ }
+
+ if (command->meta == META_SLEEP)
+ {
+ /*
+ * A \sleep doesn't execute anything, we just get the
+ * delay from the argument, and enter the CSTATE_SLEEP
+ * state. (The per-command latency will be recorded
+ * in CSTATE_SLEEP state, not here, after the delay
+ * has elapsed.)
+ */
+ int usec;
+
+ if (!evaluateSleep(st, argc, argv, &usec))
+ {
+ commandFailed(st, "sleep", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
+ st->state = CSTATE_SLEEP;
+ return now;
+ }
+ else if (command->meta == META_SET ||
+ command->meta == META_IF ||
+ command->meta == META_ELIF)
+ {
+ /* backslash commands with an expression to evaluate */
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+
+ if (command->meta == META_ELIF &&
+ conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
+ {
+ /*
+ * elif after executed block, skip eval and wait
+ * for endif
+ */
+ conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
+ goto move_to_end_command;
+ }
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ if (command->meta == META_SET)
+ {
+ if (!putVariableValue(st, argv[0], argv[1], &result))
+ {
+ commandFailed(st, "set", "assignment of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ }
+ else /* if and elif evaluated cases */
+ {
+ bool cond = valueTruth(&result);
+
+ /* execute or not depending on evaluated condition */
+ if (command->meta == META_IF)
+ {
+ conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ else /* elif */
+ {
+ /* we should get here only if the "elif" needed evaluation */
+ Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
+ conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ }
+ }
+ else if (command->meta == META_ELSE)
+ {
+ switch (conditional_stack_peek(st->cstack))
+ {
+ case IFSTATE_TRUE:
+ conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
+ break;
+ case IFSTATE_FALSE: /* inconsistent if active */
+ case IFSTATE_IGNORED: /* inconsistent if active */
+ case IFSTATE_NONE: /* else without if */
+ case IFSTATE_ELSE_TRUE: /* else after else */
+ case IFSTATE_ELSE_FALSE: /* else after else */
+ default:
+ /* dead code if conditional check is ok */
+ Assert(false);
+ }
+ goto move_to_end_command;
+ }
+ else if (command->meta == META_ENDIF)
+ {
+ Assert(!conditional_stack_empty(st->cstack));
+ conditional_stack_pop(st->cstack);
+ goto move_to_end_command;
+ }
+ else if (command->meta == META_SETSHELL)
+ {
+ if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
+ {
+ commandFailed(st, "setshell", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ /* else success */
+ }
+ else if (command->meta == META_SHELL)
+ {
+ if (!runShellCommand(st, NULL, argv + 1, argc - 1))
+ {
+ commandFailed(st, "shell", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ /* else success */
+ }
+
+move_to_end_command:
+
+ /*
+ * executing the expression or shell command might take a
+ * non-negligible amount of time; reset 'now' so that it's
+ * recomputed by caller.
+ */
+ INSTR_TIME_SET_ZERO(now);
+
+ st->state = CSTATE_END_COMMAND;
+ }
+
+ return now;
+}
+
+/*
* Print log entry after completing one transaction.
*
* We print Unix-epoch timestamps in the log, so that entries can be
@@ -3544,8 +3512,7 @@ processXactStats(TState *thread, CState *st, instr_time *now,
if (detailed && !skipped)
{
- if (INSTR_TIME_IS_ZERO(*now))
- INSTR_TIME_SET_CURRENT(*now);
+ INSTR_TIME_SET_CURRENT_LAZY(*now);
/* compute latency & lag */
latency = INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled;
@@ -5809,7 +5776,7 @@ threadRun(void *arg)
if (!is_connect)
{
- /* make connections to the database */
+ /* make connections to the database before starting */
for (i = 0; i < nstate; i++)
{
if ((state[i].con = doConnect()) == NULL)
@@ -5845,14 +5812,7 @@ threadRun(void *arg)
{
CState *st = &state[i];
- if (st->state == CSTATE_THROTTLE && timer_exceeded)
- {
- /* interrupt client that has not started a transaction */
- st->state = CSTATE_FINISHED;
- finishCon(st);
- remains--;
- }
- else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
+ if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
{
/* a nap from the script, or under throttling */
int64 this_usec;
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index f968444671..53d9699015 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -20,6 +20,8 @@
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
*
+ * INSTR_TIME_SET_CURRENT_LAZY(t) set t to current time if t is zero
+ *
* INSTR_TIME_ADD(x, y) x += y
*
* INSTR_TIME_SUBTRACT(x, y) x -= y
@@ -245,4 +247,11 @@ GetTimerFrequency(void)
#endif /* WIN32 */
+
+#define INSTR_TIME_SET_CURRENT_LAZY(t) \
+ do { \
+ if (INSTR_TIME_IS_ZERO(t)) \
+ INSTR_TIME_SET_CURRENT(t); \
+ } while (0)
+
#endif /* INSTR_TIME_H */
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
@ 2018-11-20 13:41 ` Fabien COELHO <[email protected]>
2018-11-20 14:27 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 22:48 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-21 18:08 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
0 siblings, 3 replies; 25+ messages in thread
From: Fabien COELHO @ 2018-11-20 13:41 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
Hello Alvaro,
Thanks for the review and improvements.
>> Attached v4 is a rebase after 409231919443984635b7ae9b7e2e261ab984eb1e
>
> Attached v5. I thought that separating the part that executes the
> command was an obvious readability improvement.
Hmm. It is somehow, but the aim of the refactoring is to make *ALL* state
transitions to happen in doCustom's switch (st->state) and nowhere else,
which is defeated by creating the separate function.
Although it improves readability at one level, it does not help figuring
out what happens to states, which is my primary concern: The idea is that
reading doCustom is enough to build and check the automaton, which I had
to do repeatedly while reviewing Marina's patches.
Also the added doCustom comment, which announces this property becomes
false under the refactoring function:
All state changes are performed within this function called by threadRun.
So I would suggest not to create this function.
> Do we really use the word "move" to talk about state changes? It sounds
> very odd to me. I would change that to "transition" -- would anybody
> object to that? (Not changed in v5.)
Yep. I removed the goto:-)
> On INSTR_TIME_SET_CURRENT_LAZY(), you cannot just put an "if" inside a
> macro -- consider this:
> if (foo)
> INSTR_TIME_SET_CURRENT_LAZY(bar);
> else
> something_else();
> Which "if" is the else now attached to? Now maybe the C standard has an
> answer for that (I don't know what it is), but it's hard to read and
> likely the compiler will complain anyway. I wrapped it in "do { }
> while(0)" as is customary.
Indeed, good catch.
In the attached patched, I have I think included all your changes *but*
the separate function, for the reason discussed above, and removed the
"goto".
--
Fabien.
Attachments:
[text/x-diff] pgbench-state-change-6.patch (18.4K, ../../alpine.DEB.2.21.1811201036350.7257@lancre/2-pgbench-state-change-6.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 73d3de0677..82cbd20420 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -294,24 +294,32 @@ typedef enum
{
/*
* The client must first choose a script to execute. Once chosen, it can
- * either be throttled (state CSTATE_START_THROTTLE under --rate) or start
- * right away (state CSTATE_START_TX).
+ * either be throttled (state CSTATE_START_THROTTLE under --rate), start
+ * right away (state CSTATE_START_TX) or not start at all if the timer was
+ * exceeded (state CSTATE_FINISHED).
*/
CSTATE_CHOOSE_SCRIPT,
/*
* In CSTATE_START_THROTTLE state, we calculate when to begin the next
* transaction, and advance to CSTATE_THROTTLE. CSTATE_THROTTLE state
- * sleeps until that moment. (If throttling is not enabled, doCustom()
- * falls directly through from CSTATE_START_THROTTLE to CSTATE_START_TX.)
+ * sleeps until that moment.
+ *
+ * It may also detect that the next transaction would start beyond the end
+ * of run, and switch to CSTATE_FINISHED.
*/
CSTATE_START_THROTTLE,
CSTATE_THROTTLE,
/*
* CSTATE_START_TX performs start-of-transaction processing. Establishes
- * a new connection for the transaction, in --connect mode, and records
- * the transaction start time.
+ * a new connection for the transaction in --connect mode, records
+ * the transaction start time, and proceed to the first command.
+ *
+ * Note: once a script is started, it will either error or run till
+ * its end, where it may be interrupted. It is not interrupted while
+ * running, so pgbench --time is to be understood as tx are allowed to
+ * start in that time, and will finish when their work is completed.
*/
CSTATE_START_TX,
@@ -324,9 +332,6 @@ typedef enum
* and we enter the CSTATE_SLEEP state to wait for it to expire. Other
* meta-commands are executed immediately.
*
- * CSTATE_SKIP_COMMAND for conditional branches which are not executed,
- * quickly skip commands that do not need any evaluation.
- *
* CSTATE_WAIT_RESULT waits until we get a result set back from the server
* for the current command.
*
@@ -334,19 +339,25 @@ typedef enum
*
* CSTATE_END_COMMAND records the end-of-command timestamp, increments the
* command counter, and loops back to CSTATE_START_COMMAND state.
+ *
+ * CSTATE_SKIP_COMMAND is used by conditional branches which are not
+ * executed. It quickly skip commands that do not need any evaluation.
+ * This state can move forward several commands, till there is something
+ * to do or the end of the script.
*/
CSTATE_START_COMMAND,
- CSTATE_SKIP_COMMAND,
CSTATE_WAIT_RESULT,
CSTATE_SLEEP,
CSTATE_END_COMMAND,
+ CSTATE_SKIP_COMMAND,
/*
- * CSTATE_END_TX performs end-of-transaction processing. Calculates
- * latency, and logs the transaction. In --connect mode, closes the
- * current connection. Chooses the next script to execute and starts over
- * in CSTATE_START_THROTTLE state, or enters CSTATE_FINISHED if we have no
- * more work to do.
+ * CSTATE_END_TX performs end-of-transaction processing. It calculates
+ * latency, and logs the transaction. In --connect mode, it closes the
+ * current connection.
+ *
+ * Then either starts over in CSTATE_CHOOSE_SCRIPT, or enters CSTATE_FINISHED
+ * if we have no more work to do.
*/
CSTATE_END_TX,
@@ -2821,16 +2832,12 @@ evaluateSleep(CState *st, int argc, char **argv, int *usecs)
/*
* Advance the state machine of a connection, if possible.
+ *
+ * All state changes are performed within this function called by threadRun.
*/
static void
doCustom(TState *thread, CState *st, StatsData *agg)
{
- PGresult *res;
- Command *command;
- instr_time now;
- bool end_tx_processed = false;
- int64 wait;
-
/*
* gettimeofday() isn't free, so we get the current timestamp lazily the
* first time it's needed, and reuse the same value throughout this
@@ -2839,37 +2846,44 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* means "not set yet". Reset "now" when we execute shell commands or
* expressions, which might take a non-negligible amount of time, though.
*/
+ instr_time now;
INSTR_TIME_SET_ZERO(now);
/*
* Loop in the state machine, until we have to wait for a result from the
- * server (or have to sleep, for throttling or for \sleep).
+ * server or have to sleep for throttling or \sleep.
*
* Note: In the switch-statement below, 'break' will loop back here,
* meaning "continue in the state machine". Return is used to return to
- * the caller.
+ * the caller, giving the thread the opportunity to advance another client.
*/
for (;;)
{
+ PGresult *res;
+ Command *command;
+
switch (st->state)
{
/*
* Select transaction to run.
*/
case CSTATE_CHOOSE_SCRIPT:
-
st->use_file = chooseScript(thread);
if (debug)
fprintf(stderr, "client %d executing script \"%s\"\n", st->id,
sql_script[st->use_file].desc);
- if (throttle_delay > 0)
+ /* check stack consistency */
+ Assert(conditional_stack_empty(st->cstack));
+
+ if (timer_exceeded)
+ st->state = CSTATE_FINISHED;
+ else if (throttle_delay > 0)
st->state = CSTATE_START_THROTTLE;
else
st->state = CSTATE_START_TX;
- /* check consistency */
- Assert(conditional_stack_empty(st->cstack));
+
break;
/*
@@ -2887,21 +2901,10 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* away.
*/
Assert(throttle_delay > 0);
- wait = getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger += getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
- /*
- * stop client if next transaction is beyond pgbench end of
- * execution
- */
- if (duration > 0 && st->txn_scheduled > end_time)
- {
- st->state = CSTATE_FINISHED;
- break;
- }
-
/*
* If --latency-limit is used, and this slot is already late
* so that the transaction will miss the latency limit even if
@@ -2913,20 +2916,20 @@ doCustom(TState *thread, CState *st, StatsData *agg)
{
int64 now_us;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
now_us = INSTR_TIME_GET_MICROSEC(now);
+
while (thread->throttle_trigger < now_us - latency_limit &&
(nxacts <= 0 || st->cnt < nxacts))
{
processXactStats(thread, st, &now, true, agg);
/* next rendez-vous */
- wait = getPoissonRand(&thread->ts_throttle_rs,
- throttle_delay);
- thread->throttle_trigger += wait;
+ thread->throttle_trigger +=
+ getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
st->txn_scheduled = thread->throttle_trigger;
}
- /* stop client if -t exceeded */
+
+ /* stop client if -t was exceeded in the previous skip loop */
if (nxacts > 0 && st->cnt >= nxacts)
{
st->state = CSTATE_FINISHED;
@@ -2934,38 +2937,42 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
}
+ /*
+ * stop client if next transaction is beyond pgbench end of
+ * execution.
+ */
+ if (duration > 0 && st->txn_scheduled > end_time)
+ {
+ st->state = CSTATE_FINISHED;
+ break;
+ }
+
st->state = CSTATE_THROTTLE;
- if (debug)
- fprintf(stderr, "client %d throttling " INT64_FORMAT " us\n",
- st->id, wait);
break;
/*
* Wait until it's time to start next transaction.
*/
case CSTATE_THROTTLE:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
- /* Else done sleeping, start the transaction */
- st->state = CSTATE_START_TX;
+ /* done sleeping, but do not start transaction if we are done */
+ st->state = timer_exceeded ? CSTATE_FINISHED : CSTATE_START_TX;
break;
/* Start new transaction */
case CSTATE_START_TX:
- /*
- * Establish connection on first call, or if is_connect is
- * true.
- */
+ /* establish connection if needed, i.e. under --connect */
if (st->con == NULL)
{
instr_time start;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
start = now;
if ((st->con = doConnect()) == NULL)
{
@@ -2981,28 +2988,20 @@ doCustom(TState *thread, CState *st, StatsData *agg)
memset(st->prepared, 0, sizeof(st->prepared));
}
+ /* record transaction start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->txn_begin = now;
+
/*
- * Record transaction start time under logging, progress or
- * throttling.
+ * When not throttling, this is also the transaction's
+ * scheduled start time.
*/
- if (use_log || progress || throttle_delay || latency_limit ||
- per_script_stats)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->txn_begin = now;
-
- /*
- * When not throttling, this is also the transaction's
- * scheduled start time.
- */
- if (!throttle_delay)
- st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
- }
+ if (!throttle_delay)
+ st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
/* Begin with the first command */
- st->command = 0;
st->state = CSTATE_START_COMMAND;
+ st->command = 0;
break;
/*
@@ -3021,17 +3020,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
}
- /*
- * Record statement start time if per-command latencies are
- * requested
- */
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->stmt_begin = now;
- }
+ /* record statement start time. */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->stmt_begin = now;
+ /* execute the command */
if (command->type == SQL_COMMAND)
{
if (!sendCommand(st, command))
@@ -3074,8 +3067,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
}
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
st->state = CSTATE_SLEEP;
break;
@@ -3093,10 +3086,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
{
/*
* elif after executed block, skip eval and wait
- * for endif
+ * for endif. This is a shortcut.
*/
conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
- goto move_to_end_command;
+ st->state = CSTATE_END_COMMAND;
+ break;
}
if (!evaluateExpr(thread, st, expr, &result))
@@ -3126,10 +3120,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
else /* elif */
{
- /*
- * we should get here only if the "elif"
- * needed evaluation
- */
+ /* we should get here only if the "elif" needed evaluation */
Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
}
@@ -3151,56 +3142,33 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* dead code if conditional check is ok */
Assert(false);
}
- goto move_to_end_command;
}
else if (command->meta == META_ENDIF)
{
Assert(!conditional_stack_empty(st->cstack));
conditional_stack_pop(st->cstack);
- goto move_to_end_command;
}
else if (command->meta == META_SETSHELL)
{
- bool ret = runShellCommand(st, argv[1], argv + 2, argc - 2);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
+ if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
{
commandFailed(st, "setshell", "execution of meta-command failed");
st->state = CSTATE_ABORTED;
break;
}
- else
- {
- /* succeeded */
- }
+ /* else success */
}
else if (command->meta == META_SHELL)
{
- bool ret = runShellCommand(st, NULL, argv + 1, argc - 1);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
+ if (!runShellCommand(st, NULL, argv + 1, argc - 1))
{
commandFailed(st, "shell", "execution of meta-command failed");
st->state = CSTATE_ABORTED;
break;
}
- else
- {
- /* succeeded */
- }
+ /* else success */
}
-
- move_to_end_command:
+ /* else nothing: all meta commands have been managed */
/*
* executing the expression or shell command might take a
@@ -3299,6 +3267,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
if (st->state != CSTATE_SKIP_COMMAND)
+ /* out of quick skip command loop */
break;
}
break;
@@ -3348,10 +3317,9 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* instead of CSTATE_START_TX.
*/
case CSTATE_SLEEP:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
if (INSTR_TIME_GET_MICROSEC(now) < st->sleep_until)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
/* Else done sleeping. */
st->state = CSTATE_END_COMMAND;
break;
@@ -3366,17 +3334,13 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* in thread-local data structure, if per-command latencies
* are requested.
*/
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
- /* XXX could use a mutex here, but we choose not to */
- command = sql_script[st->use_file].commands[st->command];
- addToSimpleStats(&command->stats,
- INSTR_TIME_GET_DOUBLE(now) -
- INSTR_TIME_GET_DOUBLE(st->stmt_begin));
- }
+ /* XXX could use a mutex here, but we choose not to */
+ command = sql_script[st->use_file].commands[st->command];
+ addToSimpleStats(&command->stats,
+ INSTR_TIME_GET_DOUBLE(now) -
+ INSTR_TIME_GET_DOUBLE(st->stmt_begin));
/* Go ahead with next command, to be executed or skipped */
st->command++;
@@ -3385,19 +3349,15 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
/*
- * End of transaction.
+ * End of transaction (end of script, really).
*/
case CSTATE_END_TX:
/* transaction finished: calculate latency and do log */
processXactStats(thread, st, &now, false, agg);
- /* conditional stack must be empty */
- if (!conditional_stack_empty(st->cstack))
- {
- fprintf(stderr, "end of script reached within a conditional, missing \\endif\n");
- exit(1);
- }
+ /* missing \endif... cannot happen if CheckConditional was okay */
+ Assert(conditional_stack_empty(st->cstack));
if (is_connect)
{
@@ -3411,26 +3371,17 @@ doCustom(TState *thread, CState *st, StatsData *agg)
st->state = CSTATE_FINISHED;
break;
}
+ else
+ {
+ /* next transaction */
+ st->state = CSTATE_CHOOSE_SCRIPT;
- /*
- * No transaction is underway anymore.
- */
- st->state = CSTATE_CHOOSE_SCRIPT;
-
- /*
- * If we paced through all commands in the script in this
- * loop, without returning to the caller even once, do it now.
- * This gives the thread a chance to process other
- * connections, and to do progress reporting. This can
- * currently only happen if the script consists entirely of
- * meta-commands.
- */
- if (end_tx_processed)
+ /*
+ * Ensure that we always return on this point, so as
+ * to avoid an infinite loop if the script only contains
+ * meta commands.
+ */
return;
- else
- {
- end_tx_processed = true;
- break;
}
/*
@@ -3544,8 +3495,7 @@ processXactStats(TState *thread, CState *st, instr_time *now,
if (detailed && !skipped)
{
- if (INSTR_TIME_IS_ZERO(*now))
- INSTR_TIME_SET_CURRENT(*now);
+ INSTR_TIME_SET_CURRENT_LAZY(*now);
/* compute latency & lag */
latency = INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled;
@@ -5809,7 +5759,7 @@ threadRun(void *arg)
if (!is_connect)
{
- /* make connections to the database */
+ /* make connections to the database before starting */
for (i = 0; i < nstate; i++)
{
if ((state[i].con = doConnect()) == NULL)
@@ -5845,14 +5795,7 @@ threadRun(void *arg)
{
CState *st = &state[i];
- if (st->state == CSTATE_THROTTLE && timer_exceeded)
- {
- /* interrupt client that has not started a transaction */
- st->state = CSTATE_FINISHED;
- finishCon(st);
- remains--;
- }
- else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
+ if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
{
/* a nap from the script, or under throttling */
int64 this_usec;
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index f968444671..db271c2822 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -20,6 +20,8 @@
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
*
+ * INSTR_TIME_SET_CURRENT_LAZY(t) set t to current time if t is zero
+ *
* INSTR_TIME_ADD(x, y) x += y
*
* INSTR_TIME_SUBTRACT(x, y) x -= y
@@ -245,4 +247,12 @@ GetTimerFrequency(void)
#endif /* WIN32 */
+/* same macro on all platforms */
+
+#define INSTR_TIME_SET_CURRENT_LAZY(t) \
+ do { \
+ if (INSTR_TIME_IS_ZERO(t)) \
+ INSTR_TIME_SET_CURRENT(t); \
+ } while (0)
+
#endif /* INSTR_TIME_H */
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
@ 2018-11-20 14:27 ` Alvaro Herrera <[email protected]>
2018-11-20 15:21 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2 siblings, 1 reply; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-20 14:27 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
On 2018-Nov-20, Fabien COELHO wrote:
> Hmm. It is somehow, but the aim of the refactoring is to make *ALL* state
> transitions to happen in doCustom's switch (st->state) and nowhere else,
> which is defeated by creating the separate function.
>
> Although it improves readability at one level, it does not help figuring out
> what happens to states, which is my primary concern: The idea is that
> reading doCustom is enough to build and check the automaton, which I had to
> do repeatedly while reviewing Marina's patches.
Yeah, there are conflicting goals here.
I didn't quite understand this hunk. Why does it remove the
is_latencies conditional? (The preceding comment shown here should be
updated obviously if this change is correct, but I'm not sure it is.)
@@ -3364,42 +3334,34 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/*
* command completed: accumulate per-command execution times
* in thread-local data structure, if per-command latencies
* are requested.
*/
- if (is_latencies)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
- /* XXX could use a mutex here, but we choose not to */
- command = sql_script[st->use_file].commands[st->command];
- addToSimpleStats(&command->stats,
- INSTR_TIME_GET_DOUBLE(now) -
- INSTR_TIME_GET_DOUBLE(st->stmt_begin));
- }
+ /* XXX could use a mutex here, but we choose not to */
+ command = sql_script[st->use_file].commands[st->command];
+ addToSimpleStats(&command->stats,
+ INSTR_TIME_GET_DOUBLE(now) -
+ INSTR_TIME_GET_DOUBLE(st->stmt_begin));
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-20 14:27 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
@ 2018-11-20 15:21 ` Fabien COELHO <[email protected]>
2018-11-20 22:43 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Fabien COELHO @ 2018-11-20 15:21 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
> I didn't quite understand this hunk. Why does it remove the
> is_latencies conditional? (The preceding comment shown here should be
> updated obviously if this change is correct, but I'm not sure it is.)
Pgbench runs benches a collects performance data about it.
I simplified the code to always collect data, without trying to be clever
about cases where these data may not be useful so some collection can be
skipped.
Here the test avoids recording the statement start time, mostly a simple
assignment and then later another test avoids recording the stats in the
same case, which are mostly a few adds.
ISTM that this is over optimization and unlikely to be have any measurable
effects compared to the other tasks performed when executing commands, so
a simpler code is better.
--
Fabien.
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-20 14:27 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 15:21 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
@ 2018-11-20 22:43 ` Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-20 22:43 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
On 2018-Nov-20, Fabien COELHO wrote:
>
> > I didn't quite understand this hunk. Why does it remove the
> > is_latencies conditional? (The preceding comment shown here should be
> > updated obviously if this change is correct, but I'm not sure it is.)
>
> Pgbench runs benches a collects performance data about it.
>
> I simplified the code to always collect data, without trying to be clever
> about cases where these data may not be useful so some collection can be
> skipped.
>
> Here the test avoids recording the statement start time, mostly a simple
> assignment and then later another test avoids recording the stats in the
> same case, which are mostly a few adds.
>
> ISTM that this is over optimization and unlikely to be have any measurable
> effects compared to the other tasks performed when executing commands, so a
> simpler code is better.
I don't think we're quite ready to buy this argument just yet.
See https://www.postgresql.org/message-id/flat/[email protected]
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
@ 2018-11-20 22:48 ` Alvaro Herrera <[email protected]>
2018-11-21 19:37 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2 siblings, 1 reply; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-20 22:48 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
On 2018-Nov-20, Fabien COELHO wrote:
> Hmm. It is somehow, but the aim of the refactoring is to make *ALL* state
> transitions to happen in doCustom's switch (st->state) and nowhere else,
> which is defeated by creating the separate function.
>
> Although it improves readability at one level, it does not help figuring out
> what happens to states, which is my primary concern: The idea is that
> reading doCustom is enough to build and check the automaton, which I had to
> do repeatedly while reviewing Marina's patches.
I adopted your patch, then while going over it I decided to go with my
separation proposal anyway, and re-did it.
Looking at the state of the code before this patch, I totally understand
that you want to get away from the current state of affairs. However, I
don't think my proposal makes matters worse; actually it makes them
better. Please give the code a honest look and think whether the
separation of machine state advances is really worse with my proposal.
I also renamed some things. doCustom() was a pretty bad name, as was
CSTATE_START_THROTTLE.
> Also the added doCustom comment, which announces this property becomes false
> under the refactoring function:
>
> All state changes are performed within this function called by threadRun.
>
> So I would suggest not to create this function.
I agree the state advances in threadRun were real messy.
Thanks for getting rid of the goto.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
[text/x-diff] pgbench-state-change-7.patch (29.7K, ../../[email protected]/2-pgbench-state-change-7.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 73d3de0677..710130b022 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -294,24 +294,32 @@ typedef enum
{
/*
* The client must first choose a script to execute. Once chosen, it can
- * either be throttled (state CSTATE_START_THROTTLE under --rate) or start
- * right away (state CSTATE_START_TX).
+ * either be throttled (state CSTATE_PREPARE_THROTTLE under --rate), start
+ * right away (state CSTATE_START_TX) or not start at all if the timer was
+ * exceeded (state CSTATE_FINISHED).
*/
CSTATE_CHOOSE_SCRIPT,
/*
- * In CSTATE_START_THROTTLE state, we calculate when to begin the next
+ * In CSTATE_PREPARE_THROTTLE state, we calculate when to begin the next
* transaction, and advance to CSTATE_THROTTLE. CSTATE_THROTTLE state
- * sleeps until that moment. (If throttling is not enabled, doCustom()
- * falls directly through from CSTATE_START_THROTTLE to CSTATE_START_TX.)
+ * sleeps until that moment.
+ *
+ * It may also detect that the next transaction would start beyond the end
+ * of run, and switch to CSTATE_FINISHED.
*/
- CSTATE_START_THROTTLE,
+ CSTATE_PREPARE_THROTTLE,
CSTATE_THROTTLE,
/*
* CSTATE_START_TX performs start-of-transaction processing. Establishes
- * a new connection for the transaction, in --connect mode, and records
- * the transaction start time.
+ * a new connection for the transaction in --connect mode, records the
+ * transaction start time, and proceed to the first command.
+ *
+ * Note: once a script is started, it will either error or run till its
+ * end, where it may be interrupted. It is not interrupted while running,
+ * so pgbench --time is to be understood as tx are allowed to start in
+ * that time, and will finish when their work is completed.
*/
CSTATE_START_TX,
@@ -324,9 +332,6 @@ typedef enum
* and we enter the CSTATE_SLEEP state to wait for it to expire. Other
* meta-commands are executed immediately.
*
- * CSTATE_SKIP_COMMAND for conditional branches which are not executed,
- * quickly skip commands that do not need any evaluation.
- *
* CSTATE_WAIT_RESULT waits until we get a result set back from the server
* for the current command.
*
@@ -334,19 +339,25 @@ typedef enum
*
* CSTATE_END_COMMAND records the end-of-command timestamp, increments the
* command counter, and loops back to CSTATE_START_COMMAND state.
+ *
+ * CSTATE_SKIP_COMMAND is used by conditional branches which are not
+ * executed. It quickly skip commands that do not need any evaluation.
+ * This state can move forward several commands, till there is something
+ * to do or the end of the script.
*/
CSTATE_START_COMMAND,
- CSTATE_SKIP_COMMAND,
CSTATE_WAIT_RESULT,
CSTATE_SLEEP,
CSTATE_END_COMMAND,
+ CSTATE_SKIP_COMMAND,
/*
- * CSTATE_END_TX performs end-of-transaction processing. Calculates
- * latency, and logs the transaction. In --connect mode, closes the
- * current connection. Chooses the next script to execute and starts over
- * in CSTATE_START_THROTTLE state, or enters CSTATE_FINISHED if we have no
- * more work to do.
+ * CSTATE_END_TX performs end-of-transaction processing. It calculates
+ * latency, and logs the transaction. In --connect mode, it closes the
+ * current connection.
+ *
+ * Then either starts over in CSTATE_CHOOSE_SCRIPT, or enters
+ * CSTATE_FINISHED if we have no more work to do.
*/
CSTATE_END_TX,
@@ -567,7 +578,10 @@ static void setNullValue(PgBenchValue *pv);
static void setBoolValue(PgBenchValue *pv, bool bval);
static void setIntValue(PgBenchValue *pv, int64 ival);
static void setDoubleValue(PgBenchValue *pv, double dval);
-static bool evaluateExpr(TState *, CState *, PgBenchExpr *, PgBenchValue *);
+static bool evaluateExpr(TState *thread, CState *st, PgBenchExpr *expr,
+ PgBenchValue *retval);
+static instr_time doExecuteCommand(TState *thread, CState *st,
+ instr_time now);
static void doLog(TState *thread, CState *st,
StatsData *agg, bool skipped, double latency, double lag);
static void processXactStats(TState *thread, CState *st, instr_time *now,
@@ -2820,16 +2834,12 @@ evaluateSleep(CState *st, int argc, char **argv, int *usecs)
}
/*
- * Advance the state machine of a connection, if possible.
+ * Advance the state machine of a connection.
*/
static void
-doCustom(TState *thread, CState *st, StatsData *agg)
+advanceConnectionState(TState *thread, CState *st, StatsData *agg)
{
- PGresult *res;
- Command *command;
instr_time now;
- bool end_tx_processed = false;
- int64 wait;
/*
* gettimeofday() isn't free, so we get the current timestamp lazily the
@@ -2843,129 +2853,45 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/*
* Loop in the state machine, until we have to wait for a result from the
- * server (or have to sleep, for throttling or for \sleep).
+ * server or have to sleep for throttling or \sleep.
*
* Note: In the switch-statement below, 'break' will loop back here,
* meaning "continue in the state machine". Return is used to return to
- * the caller.
+ * the caller, giving the thread the opportunity to advance another
+ * client.
*/
for (;;)
{
+ PGresult *res;
+
switch (st->state)
{
- /*
- * Select transaction to run.
- */
+ /* Select transaction (script) to run. */
case CSTATE_CHOOSE_SCRIPT:
-
st->use_file = chooseScript(thread);
+ Assert(conditional_stack_empty(st->cstack));
if (debug)
fprintf(stderr, "client %d executing script \"%s\"\n", st->id,
sql_script[st->use_file].desc);
- if (throttle_delay > 0)
- st->state = CSTATE_START_THROTTLE;
- else
- st->state = CSTATE_START_TX;
- /* check consistency */
- Assert(conditional_stack_empty(st->cstack));
+ /*
+ * If time is over, we're done; otherwise, get ready to start
+ * a new transaction, or to get throttled if that's requested.
+ */
+ st->state = timer_exceeded ? CSTATE_FINISHED :
+ throttle_delay > 0 ? CSTATE_PREPARE_THROTTLE : CSTATE_START_TX;
break;
- /*
- * Handle throttling once per transaction by sleeping.
- */
- case CSTATE_START_THROTTLE:
-
- /*
- * Generate a delay such that the series of delays will
- * approximate a Poisson distribution centered on the
- * throttle_delay time.
- *
- * If transactions are too slow or a given wait is shorter
- * than a transaction, the next transaction will start right
- * away.
- */
- Assert(throttle_delay > 0);
- wait = getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
-
- thread->throttle_trigger += wait;
- st->txn_scheduled = thread->throttle_trigger;
-
- /*
- * stop client if next transaction is beyond pgbench end of
- * execution
- */
- if (duration > 0 && st->txn_scheduled > end_time)
- {
- st->state = CSTATE_FINISHED;
- break;
- }
-
- /*
- * If --latency-limit is used, and this slot is already late
- * so that the transaction will miss the latency limit even if
- * it completed immediately, we skip this time slot and
- * iterate till the next slot that isn't late yet. But don't
- * iterate beyond the -t limit, if one is given.
- */
- if (latency_limit)
- {
- int64 now_us;
-
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- now_us = INSTR_TIME_GET_MICROSEC(now);
- while (thread->throttle_trigger < now_us - latency_limit &&
- (nxacts <= 0 || st->cnt < nxacts))
- {
- processXactStats(thread, st, &now, true, agg);
- /* next rendez-vous */
- wait = getPoissonRand(&thread->ts_throttle_rs,
- throttle_delay);
- thread->throttle_trigger += wait;
- st->txn_scheduled = thread->throttle_trigger;
- }
- /* stop client if -t exceeded */
- if (nxacts > 0 && st->cnt >= nxacts)
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- }
-
- st->state = CSTATE_THROTTLE;
- if (debug)
- fprintf(stderr, "client %d throttling " INT64_FORMAT " us\n",
- st->id, wait);
- break;
-
- /*
- * Wait until it's time to start next transaction.
- */
- case CSTATE_THROTTLE:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
- return; /* Still sleeping, nothing to do here */
-
- /* Else done sleeping, start the transaction */
- st->state = CSTATE_START_TX;
- break;
-
- /* Start new transaction */
+ /* Start new transaction (script) */
case CSTATE_START_TX:
- /*
- * Establish connection on first call, or if is_connect is
- * true.
- */
+ /* establish connection if needed, i.e. under --connect */
if (st->con == NULL)
{
instr_time start;
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
start = now;
if ((st->con = doConnect()) == NULL)
{
@@ -2981,235 +2907,126 @@ doCustom(TState *thread, CState *st, StatsData *agg)
memset(st->prepared, 0, sizeof(st->prepared));
}
- /*
- * Record transaction start time under logging, progress or
- * throttling.
- */
- if (use_log || progress || throttle_delay || latency_limit ||
- per_script_stats)
- {
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->txn_begin = now;
+ /* record transaction start time */
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ st->txn_begin = now;
- /*
- * When not throttling, this is also the transaction's
- * scheduled start time.
- */
- if (!throttle_delay)
- st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
- }
+ /*
+ * When not throttling, this is also the transaction's
+ * scheduled start time.
+ */
+ if (!throttle_delay)
+ st->txn_scheduled = INSTR_TIME_GET_MICROSEC(now);
/* Begin with the first command */
- st->command = 0;
st->state = CSTATE_START_COMMAND;
+ st->command = 0;
+ break;
+
+ /*
+ * Handle throttling once per transaction by sleeping.
+ */
+ case CSTATE_PREPARE_THROTTLE:
+
+ /*
+ * Generate a delay such that the series of delays will
+ * approximate a Poisson distribution centered on the
+ * throttle_delay time.
+ *
+ * If transactions are too slow or a given wait is shorter
+ * than a transaction, the next transaction will start right
+ * away.
+ */
+ Assert(throttle_delay > 0);
+
+ thread->throttle_trigger +=
+ getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
+ st->txn_scheduled = thread->throttle_trigger;
+
+ /*
+ * If --latency-limit is used, and this slot is already late
+ * so that the transaction will miss the latency limit even if
+ * it completed immediately, skip this time slot and schedule
+ * to continue running on the next slot that isn't late yet.
+ * But don't iterate beyond the -t limit, if one is given.
+ */
+ if (latency_limit)
+ {
+ int64 now_us;
+
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+ now_us = INSTR_TIME_GET_MICROSEC(now);
+
+ while (thread->throttle_trigger < now_us - latency_limit &&
+ (nxacts <= 0 || st->cnt < nxacts))
+ {
+ processXactStats(thread, st, &now, true, agg);
+ /* next rendez-vous */
+ thread->throttle_trigger +=
+ getPoissonRand(&thread->ts_throttle_rs, throttle_delay);
+ st->txn_scheduled = thread->throttle_trigger;
+ }
+
+ /*
+ * stop client if -t was exceeded in the previous skip
+ * loop
+ */
+ if (nxacts > 0 && st->cnt >= nxacts)
+ {
+ st->state = CSTATE_FINISHED;
+ break;
+ }
+ }
+
+ /*
+ * stop client if next transaction is beyond pgbench end of
+ * execution; otherwise, throttle it.
+ */
+ st->state = end_time > 0 && st->txn_scheduled > end_time ?
+ CSTATE_FINISHED : CSTATE_THROTTLE;
+ break;
+
+ /*
+ * Wait until it's time to start next transaction.
+ */
+ case CSTATE_THROTTLE:
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
+ if (INSTR_TIME_GET_MICROSEC(now) < st->txn_scheduled)
+ return; /* still sleeping, nothing to do here */
+
+ /* done sleeping, but don't start transaction if we're done */
+ st->state = timer_exceeded ? CSTATE_FINISHED : CSTATE_START_TX;
break;
/*
* Send a command to server (or execute a meta-command)
*/
case CSTATE_START_COMMAND:
- command = sql_script[st->use_file].commands[st->command];
-
- /*
- * If we reached the end of the script, move to end-of-xact
- * processing.
- */
- if (command == NULL)
+ /* Transition to script end processing if done */
+ if (sql_script[st->use_file].commands[st->command] == NULL)
{
st->state = CSTATE_END_TX;
break;
}
- /*
- * Record statement start time if per-command latencies are
- * requested
- */
+ /* record begin time of next command, and initiate it */
if (is_latencies)
{
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
st->stmt_begin = now;
}
+ now = doExecuteCommand(thread, st, now);
- if (command->type == SQL_COMMAND)
- {
- if (!sendCommand(st, command))
- {
- commandFailed(st, "SQL", "SQL command send failed");
- st->state = CSTATE_ABORTED;
- }
- else
- st->state = CSTATE_WAIT_RESULT;
- }
- else if (command->type == META_COMMAND)
- {
- int argc = command->argc,
- i;
- char **argv = command->argv;
-
- if (debug)
- {
- fprintf(stderr, "client %d executing \\%s", st->id, argv[0]);
- for (i = 1; i < argc; i++)
- fprintf(stderr, " %s", argv[i]);
- fprintf(stderr, "\n");
- }
-
- if (command->meta == META_SLEEP)
- {
- /*
- * A \sleep doesn't execute anything, we just get the
- * delay from the argument, and enter the CSTATE_SLEEP
- * state. (The per-command latency will be recorded
- * in CSTATE_SLEEP state, not here, after the delay
- * has elapsed.)
- */
- int usec;
-
- if (!evaluateSleep(st, argc, argv, &usec))
- {
- commandFailed(st, "sleep", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
-
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
- st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
- st->state = CSTATE_SLEEP;
- break;
- }
- else if (command->meta == META_SET ||
- command->meta == META_IF ||
- command->meta == META_ELIF)
- {
- /* backslash commands with an expression to evaluate */
- PgBenchExpr *expr = command->expr;
- PgBenchValue result;
-
- if (command->meta == META_ELIF &&
- conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
- {
- /*
- * elif after executed block, skip eval and wait
- * for endif
- */
- conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
- goto move_to_end_command;
- }
-
- if (!evaluateExpr(thread, st, expr, &result))
- {
- commandFailed(st, argv[0], "evaluation of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
-
- if (command->meta == META_SET)
- {
- if (!putVariableValue(st, argv[0], argv[1], &result))
- {
- commandFailed(st, "set", "assignment of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- }
- else /* if and elif evaluated cases */
- {
- bool cond = valueTruth(&result);
-
- /* execute or not depending on evaluated condition */
- if (command->meta == META_IF)
- {
- conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- else /* elif */
- {
- /*
- * we should get here only if the "elif"
- * needed evaluation
- */
- Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
- conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- }
- }
- else if (command->meta == META_ELSE)
- {
- switch (conditional_stack_peek(st->cstack))
- {
- case IFSTATE_TRUE:
- conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
- break;
- case IFSTATE_FALSE: /* inconsistent if active */
- case IFSTATE_IGNORED: /* inconsistent if active */
- case IFSTATE_NONE: /* else without if */
- case IFSTATE_ELSE_TRUE: /* else after else */
- case IFSTATE_ELSE_FALSE: /* else after else */
- default:
- /* dead code if conditional check is ok */
- Assert(false);
- }
- goto move_to_end_command;
- }
- else if (command->meta == META_ENDIF)
- {
- Assert(!conditional_stack_empty(st->cstack));
- conditional_stack_pop(st->cstack);
- goto move_to_end_command;
- }
- else if (command->meta == META_SETSHELL)
- {
- bool ret = runShellCommand(st, argv[1], argv + 2, argc - 2);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
- {
- commandFailed(st, "setshell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- else
- {
- /* succeeded */
- }
- }
- else if (command->meta == META_SHELL)
- {
- bool ret = runShellCommand(st, NULL, argv + 1, argc - 1);
-
- if (timer_exceeded) /* timeout */
- {
- st->state = CSTATE_FINISHED;
- break;
- }
- else if (!ret) /* on error */
- {
- commandFailed(st, "shell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- break;
- }
- else
- {
- /* succeeded */
- }
- }
-
- move_to_end_command:
-
- /*
- * executing the expression or shell command might take a
- * non-negligible amount of time, so reset 'now'
- */
- INSTR_TIME_SET_ZERO(now);
-
- st->state = CSTATE_END_COMMAND;
- }
+ /*
+ * We're now waiting for an SQL command to complete, or
+ * finished processing a metacommand, or need to sleep, or
+ * something bad happened.
+ */
+ Assert(st->state == CSTATE_WAIT_RESULT ||
+ st->state == CSTATE_END_COMMAND ||
+ st->state == CSTATE_SLEEP ||
+ st->state == CSTATE_ABORTED);
break;
/*
@@ -3220,6 +3037,8 @@ doCustom(TState *thread, CState *st, StatsData *agg)
/* quickly skip commands until something to do... */
while (true)
{
+ Command *command;
+
command = sql_script[st->use_file].commands[st->command];
/* cannot reach end of script in that state */
@@ -3299,6 +3118,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
if (st->state != CSTATE_SKIP_COMMAND)
+ /* out of quick skip command loop */
break;
}
break;
@@ -3307,11 +3127,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* Wait for the current SQL command to complete
*/
case CSTATE_WAIT_RESULT:
- command = sql_script[st->use_file].commands[st->command];
if (debug)
fprintf(stderr, "client %d receiving\n", st->id);
if (!PQconsumeInput(st->con))
- { /* there's something wrong */
+ {
+ /* there's something wrong */
commandFailed(st, "SQL", "perhaps the backend died while processing");
st->state = CSTATE_ABORTED;
break;
@@ -3319,9 +3139,7 @@ doCustom(TState *thread, CState *st, StatsData *agg)
if (PQisBusy(st->con))
return; /* don't have the whole result yet */
- /*
- * Read and discard the query result;
- */
+ /* Read and discard the query result */
res = PQgetResult(st->con);
switch (PQresultStatus(res))
{
@@ -3348,10 +3166,9 @@ doCustom(TState *thread, CState *st, StatsData *agg)
* instead of CSTATE_START_TX.
*/
case CSTATE_SLEEP:
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ INSTR_TIME_SET_CURRENT_LAZY(now);
if (INSTR_TIME_GET_MICROSEC(now) < st->sleep_until)
- return; /* Still sleeping, nothing to do here */
+ return; /* still sleeping, nothing to do here */
/* Else done sleeping. */
st->state = CSTATE_END_COMMAND;
break;
@@ -3368,11 +3185,11 @@ doCustom(TState *thread, CState *st, StatsData *agg)
*/
if (is_latencies)
{
- if (INSTR_TIME_IS_ZERO(now))
- INSTR_TIME_SET_CURRENT(now);
+ Command *command = sql_script[st->use_file].commands[st->command];
+
+ INSTR_TIME_SET_CURRENT_LAZY(now);
/* XXX could use a mutex here, but we choose not to */
- command = sql_script[st->use_file].commands[st->command];
addToSimpleStats(&command->stats,
INSTR_TIME_GET_DOUBLE(now) -
INSTR_TIME_GET_DOUBLE(st->stmt_begin));
@@ -3385,19 +3202,18 @@ doCustom(TState *thread, CState *st, StatsData *agg)
break;
/*
- * End of transaction.
+ * End of transaction (end of script, really).
*/
case CSTATE_END_TX:
/* transaction finished: calculate latency and do log */
processXactStats(thread, st, &now, false, agg);
- /* conditional stack must be empty */
- if (!conditional_stack_empty(st->cstack))
- {
- fprintf(stderr, "end of script reached within a conditional, missing \\endif\n");
- exit(1);
- }
+ /*
+ * missing \endif... cannot happen if CheckConditional was
+ * okay
+ */
+ Assert(conditional_stack_empty(st->cstack));
if (is_connect)
{
@@ -3411,26 +3227,17 @@ doCustom(TState *thread, CState *st, StatsData *agg)
st->state = CSTATE_FINISHED;
break;
}
-
- /*
- * No transaction is underway anymore.
- */
- st->state = CSTATE_CHOOSE_SCRIPT;
-
- /*
- * If we paced through all commands in the script in this
- * loop, without returning to the caller even once, do it now.
- * This gives the thread a chance to process other
- * connections, and to do progress reporting. This can
- * currently only happen if the script consists entirely of
- * meta-commands.
- */
- if (end_tx_processed)
- return;
else
{
- end_tx_processed = true;
- break;
+ /* next transaction (script) */
+ st->state = CSTATE_CHOOSE_SCRIPT;
+
+ /*
+ * Ensure that we always return on this point, so as to
+ * avoid an infinite loop if the script only contains meta
+ * commands.
+ */
+ return;
}
/*
@@ -3445,6 +3252,182 @@ doCustom(TState *thread, CState *st, StatsData *agg)
}
/*
+ * Subroutine for advanceConnectionState -- execute or initiate the current
+ * command, and transition to next state appropriately.
+ *
+ * Returns an updated timestamp from 'now', used to update 'now' at callsite.
+ */
+static instr_time
+doExecuteCommand(TState *thread, CState *st, instr_time now)
+{
+ Command *command = sql_script[st->use_file].commands[st->command];
+
+ /* execute the command */
+ if (command->type == SQL_COMMAND)
+ {
+ if (!sendCommand(st, command))
+ {
+ commandFailed(st, "SQL", "SQL command send failed");
+ st->state = CSTATE_ABORTED;
+ }
+ else
+ st->state = CSTATE_WAIT_RESULT;
+ }
+ else if (command->type == META_COMMAND)
+ {
+ int argc = command->argc;
+ char **argv = command->argv;
+
+ if (debug)
+ {
+ fprintf(stderr, "client %d executing \\%s",
+ st->id, argv[0]);
+ for (int i = 1; i < argc; i++)
+ fprintf(stderr, " %s", argv[i]);
+ fprintf(stderr, "\n");
+ }
+
+ if (command->meta == META_SLEEP)
+ {
+ int usec;
+
+ /*
+ * A \sleep doesn't execute anything, we just get the delay from
+ * the argument, and enter the CSTATE_SLEEP state. (The
+ * per-command latency will be recorded in CSTATE_SLEEP state, not
+ * here, after the delay has elapsed.)
+ */
+ if (!evaluateSleep(st, argc, argv, &usec))
+ {
+ commandFailed(st, "sleep", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ INSTR_TIME_SET_CURRENT_LAZY(now);
+
+ st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
+ st->state = CSTATE_SLEEP;
+ return now;
+ }
+ else if (command->meta == META_SET)
+ {
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ if (!putVariableValue(st, argv[0], argv[1], &result))
+ {
+ commandFailed(st, "set", "assignment of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ }
+ else if (command->meta == META_IF)
+ {
+ /* backslash commands with an expression to evaluate */
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+ bool cond;
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ cond = valueTruth(&result);
+ conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ else if (command->meta == META_ELIF)
+ {
+ /* backslash commands with an expression to evaluate */
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+ bool cond;
+
+ if (conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
+ {
+ /*
+ * elif after executed block, skip eval and wait for endif.
+ */
+ conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
+ st->state = CSTATE_END_COMMAND;
+ return now;
+ }
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+
+ cond = valueTruth(&result);
+ Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
+ conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ else if (command->meta == META_ELSE)
+ {
+ switch (conditional_stack_peek(st->cstack))
+ {
+ case IFSTATE_TRUE:
+ conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
+ break;
+ case IFSTATE_FALSE: /* inconsistent if active */
+ case IFSTATE_IGNORED: /* inconsistent if active */
+ case IFSTATE_NONE: /* else without if */
+ case IFSTATE_ELSE_TRUE: /* else after else */
+ case IFSTATE_ELSE_FALSE: /* else after else */
+ default:
+ /* dead code if conditional check is ok */
+ Assert(false);
+ }
+ }
+ else if (command->meta == META_ENDIF)
+ {
+ Assert(!conditional_stack_empty(st->cstack));
+ conditional_stack_pop(st->cstack);
+ }
+ else if (command->meta == META_SETSHELL)
+ {
+ if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
+ {
+ commandFailed(st, "setshell", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ }
+ else if (command->meta == META_SHELL)
+ {
+ if (!runShellCommand(st, NULL, argv + 1, argc - 1))
+ {
+ commandFailed(st, "shell", "execution of meta-command failed");
+ st->state = CSTATE_ABORTED;
+ return now;
+ }
+ }
+
+ /*
+ * executing the expression or shell command might have taken a
+ * non-negligible amount of time, so reset 'now'
+ */
+ INSTR_TIME_SET_ZERO(now);
+
+ st->state = CSTATE_END_COMMAND;
+ }
+
+ return now;
+}
+
+/*
* Print log entry after completing one transaction.
*
* We print Unix-epoch timestamps in the log, so that entries can be
@@ -3544,8 +3527,7 @@ processXactStats(TState *thread, CState *st, instr_time *now,
if (detailed && !skipped)
{
- if (INSTR_TIME_IS_ZERO(*now))
- INSTR_TIME_SET_CURRENT(*now);
+ INSTR_TIME_SET_CURRENT_LAZY(*now);
/* compute latency & lag */
latency = INSTR_TIME_GET_MICROSEC(*now) - st->txn_scheduled;
@@ -5809,7 +5791,7 @@ threadRun(void *arg)
if (!is_connect)
{
- /* make connections to the database */
+ /* make connections to the database before starting */
for (i = 0; i < nstate; i++)
{
if ((state[i].con = doConnect()) == NULL)
@@ -5845,14 +5827,7 @@ threadRun(void *arg)
{
CState *st = &state[i];
- if (st->state == CSTATE_THROTTLE && timer_exceeded)
- {
- /* interrupt client that has not started a transaction */
- st->state = CSTATE_FINISHED;
- finishCon(st);
- remains--;
- }
- else if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
+ if (st->state == CSTATE_SLEEP || st->state == CSTATE_THROTTLE)
{
/* a nap from the script, or under throttling */
int64 this_usec;
@@ -5971,7 +5946,7 @@ threadRun(void *arg)
if (st->state == CSTATE_WAIT_RESULT)
{
- /* don't call doCustom unless data is available */
+ /* don't call advanceConnectionState unless data is available */
int sock = PQsocket(st->con);
if (sock < 0)
@@ -5991,9 +5966,12 @@ threadRun(void *arg)
continue;
}
- doCustom(thread, st, &aggs);
+ advanceConnectionState(thread, st, &aggs);
- /* If doCustom changed client to finished state, reduce remains */
+ /*
+ * If advanceConnectionState changed client to finished state,
+ * that's one less client that remains.
+ */
if (st->state == CSTATE_FINISHED || st->state == CSTATE_ABORTED)
remains--;
}
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index f968444671..db271c2822 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -20,6 +20,8 @@
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
*
+ * INSTR_TIME_SET_CURRENT_LAZY(t) set t to current time if t is zero
+ *
* INSTR_TIME_ADD(x, y) x += y
*
* INSTR_TIME_SUBTRACT(x, y) x -= y
@@ -245,4 +247,12 @@ GetTimerFrequency(void)
#endif /* WIN32 */
+/* same macro on all platforms */
+
+#define INSTR_TIME_SET_CURRENT_LAZY(t) \
+ do { \
+ if (INSTR_TIME_IS_ZERO(t)) \
+ INSTR_TIME_SET_CURRENT(t); \
+ } while (0)
+
#endif /* INSTR_TIME_H */
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-20 22:48 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
@ 2018-11-21 19:37 ` Alvaro Herrera <[email protected]>
2018-11-24 08:58 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-21 19:37 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
I just pushed this. I hope not to have upset you too much with the
subroutine thing.
Thanks for the submission and Kirk for the review.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-20 22:48 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-21 19:37 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
@ 2018-11-24 08:58 ` Fabien COELHO <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Fabien COELHO @ 2018-11-24 08:58 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
Hello Alvaro,
> I just pushed this. I hope not to have upset you too much with the
> subroutine thing.
Sorry for the feedback delay, my week was kind of overloaded…
Thanks for the push.
About the patch you committed, a post-commit review:
- the state and function renamings are indeed a good thing.
- I'm not fond of "now = func(..., now)", I'd have just passed a
reference.
- I'd put out the meta commands, but keep the SQL case and the state
assignment in the initial function, so that all state changes are in
one function… which was the initial aim of the submission.
Kind of a compromise:-)
See the attached followup patch which implements these suggestions. The
patch is quite small under "git diff -w", but larger because of the
reindentation as one "if" level is removed.
--
Fabien.
Attachments:
[text/x-diff] pgbench-state-change-followup-1.patch (11.2K, ../../alpine.DEB.2.21.1811240904500.12627@lancre/2-pgbench-state-change-followup-1.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index c64e16187a..7392cf8688 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -580,8 +580,7 @@ static void setIntValue(PgBenchValue *pv, int64 ival);
static void setDoubleValue(PgBenchValue *pv, double dval);
static bool evaluateExpr(TState *thread, CState *st, PgBenchExpr *expr,
PgBenchValue *retval);
-static instr_time doExecuteCommand(TState *thread, CState *st,
- instr_time now);
+static ConnectionStateEnum executeMetaCommand(TState *thread, CState *st, instr_time *now);
static void doLog(TState *thread, CState *st,
StatsData *agg, bool skipped, double latency, double lag);
static void processXactStats(TState *thread, CState *st, instr_time *now,
@@ -2862,6 +2861,7 @@ advanceConnectionState(TState *thread, CState *st, StatsData *agg)
*/
for (;;)
{
+ Command *command;
PGresult *res;
switch (st->state)
@@ -3003,8 +3003,10 @@ advanceConnectionState(TState *thread, CState *st, StatsData *agg)
* Send a command to server (or execute a meta-command)
*/
case CSTATE_START_COMMAND:
+ command = sql_script[st->use_file].commands[st->command];
+
/* Transition to script end processing if done */
- if (sql_script[st->use_file].commands[st->command] == NULL)
+ if (command == NULL)
{
st->state = CSTATE_END_TX;
break;
@@ -3016,7 +3018,27 @@ advanceConnectionState(TState *thread, CState *st, StatsData *agg)
INSTR_TIME_SET_CURRENT_LAZY(now);
st->stmt_begin = now;
}
- now = doExecuteCommand(thread, st, now);
+
+ if (command->type == SQL_COMMAND)
+ {
+ if (!sendCommand(st, command))
+ {
+ commandFailed(st, "SQL", "SQL command send failed");
+ st->state = CSTATE_ABORTED;
+ }
+ else
+ st->state = CSTATE_WAIT_RESULT;
+ }
+ else if (command->type == META_COMMAND)
+ {
+ /*-----
+ * Possible state changes when executing meta commands:
+ * - on errors CSTATE_ABORTED
+ * - on sleep CSTATE_SLEEP
+ * - else CSTATE_END_COMMAND
+ */
+ st->state = executeMetaCommand(thread, st, &now);
+ }
/*
* We're now waiting for an SQL command to complete, or
@@ -3254,178 +3276,151 @@ advanceConnectionState(TState *thread, CState *st, StatsData *agg)
/*
* Subroutine for advanceConnectionState -- execute or initiate the current
- * command, and transition to next state appropriately.
- *
- * Returns an updated timestamp from 'now', used to update 'now' at callsite.
+ * meta command, and return to next state appropriately.
*/
-static instr_time
-doExecuteCommand(TState *thread, CState *st, instr_time now)
+static ConnectionStateEnum
+executeMetaCommand(TState *thread, CState *st, instr_time *now)
{
Command *command = sql_script[st->use_file].commands[st->command];
+ int argc;
+ char **argv;
- /* execute the command */
- if (command->type == SQL_COMMAND)
+ Assert(command != NULL && command->type == META_COMMAND);
+
+ argc = command->argc;
+ argv = command->argv;
+
+ if (debug)
{
- if (!sendCommand(st, command))
- {
- commandFailed(st, "SQL", "SQL command send failed");
- st->state = CSTATE_ABORTED;
- }
- else
- st->state = CSTATE_WAIT_RESULT;
+ fprintf(stderr, "client %d executing \\%s", st->id, argv[0]);
+ for (int i = 1; i < argc; i++)
+ fprintf(stderr, " %s", argv[i]);
+ fprintf(stderr, "\n");
}
- else if (command->type == META_COMMAND)
+
+ if (command->meta == META_SLEEP)
{
- int argc = command->argc;
- char **argv = command->argv;
-
- if (debug)
- {
- fprintf(stderr, "client %d executing \\%s",
- st->id, argv[0]);
- for (int i = 1; i < argc; i++)
- fprintf(stderr, " %s", argv[i]);
- fprintf(stderr, "\n");
- }
-
- if (command->meta == META_SLEEP)
- {
- int usec;
-
- /*
- * A \sleep doesn't execute anything, we just get the delay from
- * the argument, and enter the CSTATE_SLEEP state. (The
- * per-command latency will be recorded in CSTATE_SLEEP state, not
- * here, after the delay has elapsed.)
- */
- if (!evaluateSleep(st, argc, argv, &usec))
- {
- commandFailed(st, "sleep", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
-
- INSTR_TIME_SET_CURRENT_LAZY(now);
-
- st->sleep_until = INSTR_TIME_GET_MICROSEC(now) + usec;
- st->state = CSTATE_SLEEP;
- return now;
- }
- else if (command->meta == META_SET)
- {
- PgBenchExpr *expr = command->expr;
- PgBenchValue result;
-
- if (!evaluateExpr(thread, st, expr, &result))
- {
- commandFailed(st, argv[0], "evaluation of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
-
- if (!putVariableValue(st, argv[0], argv[1], &result))
- {
- commandFailed(st, "set", "assignment of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
- }
- else if (command->meta == META_IF)
- {
- /* backslash commands with an expression to evaluate */
- PgBenchExpr *expr = command->expr;
- PgBenchValue result;
- bool cond;
-
- if (!evaluateExpr(thread, st, expr, &result))
- {
- commandFailed(st, argv[0], "evaluation of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
-
- cond = valueTruth(&result);
- conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- else if (command->meta == META_ELIF)
- {
- /* backslash commands with an expression to evaluate */
- PgBenchExpr *expr = command->expr;
- PgBenchValue result;
- bool cond;
-
- if (conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
- {
- /*
- * elif after executed block, skip eval and wait for endif.
- */
- conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
- st->state = CSTATE_END_COMMAND;
- return now;
- }
-
- if (!evaluateExpr(thread, st, expr, &result))
- {
- commandFailed(st, argv[0], "evaluation of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
-
- cond = valueTruth(&result);
- Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
- conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
- }
- else if (command->meta == META_ELSE)
- {
- switch (conditional_stack_peek(st->cstack))
- {
- case IFSTATE_TRUE:
- conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
- break;
- case IFSTATE_FALSE: /* inconsistent if active */
- case IFSTATE_IGNORED: /* inconsistent if active */
- case IFSTATE_NONE: /* else without if */
- case IFSTATE_ELSE_TRUE: /* else after else */
- case IFSTATE_ELSE_FALSE: /* else after else */
- default:
- /* dead code if conditional check is ok */
- Assert(false);
- }
- }
- else if (command->meta == META_ENDIF)
- {
- Assert(!conditional_stack_empty(st->cstack));
- conditional_stack_pop(st->cstack);
- }
- else if (command->meta == META_SETSHELL)
- {
- if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
- {
- commandFailed(st, "setshell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
- }
- else if (command->meta == META_SHELL)
- {
- if (!runShellCommand(st, NULL, argv + 1, argc - 1))
- {
- commandFailed(st, "shell", "execution of meta-command failed");
- st->state = CSTATE_ABORTED;
- return now;
- }
- }
+ int usec;
/*
- * executing the expression or shell command might have taken a
- * non-negligible amount of time, so reset 'now'
+ * A \sleep doesn't execute anything, we just get the delay from
+ * the argument, and enter the CSTATE_SLEEP state. (The
+ * per-command latency will be recorded in CSTATE_SLEEP state, not
+ * here, after the delay has elapsed.)
*/
- INSTR_TIME_SET_ZERO(now);
+ if (!evaluateSleep(st, argc, argv, &usec))
+ {
+ commandFailed(st, "sleep", "execution of meta-command failed");
+ return CSTATE_ABORTED;
+ }
- st->state = CSTATE_END_COMMAND;
+ INSTR_TIME_SET_CURRENT_LAZY(*now);
+ st->sleep_until = INSTR_TIME_GET_MICROSEC(*now) + usec;
+ return CSTATE_SLEEP;
}
+ else if (command->meta == META_SET)
+ {
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
- return now;
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+
+ if (!putVariableValue(st, argv[0], argv[1], &result))
+ {
+ commandFailed(st, "set", "assignment of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+ }
+ else if (command->meta == META_IF)
+ {
+ /* backslash commands with an expression to evaluate */
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+ bool cond;
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+
+ cond = valueTruth(&result);
+ conditional_stack_push(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ else if (command->meta == META_ELIF)
+ {
+ /* backslash commands with an expression to evaluate */
+ PgBenchExpr *expr = command->expr;
+ PgBenchValue result;
+ bool cond;
+
+ if (conditional_stack_peek(st->cstack) == IFSTATE_TRUE)
+ {
+ /* elif after executed block, skip eval and wait for endif. */
+ conditional_stack_poke(st->cstack, IFSTATE_IGNORED);
+ return CSTATE_END_COMMAND;
+ }
+
+ if (!evaluateExpr(thread, st, expr, &result))
+ {
+ commandFailed(st, argv[0], "evaluation of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+
+ cond = valueTruth(&result);
+ Assert(conditional_stack_peek(st->cstack) == IFSTATE_FALSE);
+ conditional_stack_poke(st->cstack, cond ? IFSTATE_TRUE : IFSTATE_FALSE);
+ }
+ else if (command->meta == META_ELSE)
+ {
+ switch (conditional_stack_peek(st->cstack))
+ {
+ case IFSTATE_TRUE:
+ conditional_stack_poke(st->cstack, IFSTATE_ELSE_FALSE);
+ break;
+ case IFSTATE_FALSE: /* inconsistent if active */
+ case IFSTATE_IGNORED: /* inconsistent if active */
+ case IFSTATE_NONE: /* else without if */
+ case IFSTATE_ELSE_TRUE: /* else after else */
+ case IFSTATE_ELSE_FALSE: /* else after else */
+ default:
+ /* dead code if conditional check is ok */
+ Assert(false);
+ }
+ }
+ else if (command->meta == META_ENDIF)
+ {
+ Assert(!conditional_stack_empty(st->cstack));
+ conditional_stack_pop(st->cstack);
+ }
+ else if (command->meta == META_SETSHELL)
+ {
+ if (!runShellCommand(st, argv[1], argv + 2, argc - 2))
+ {
+ commandFailed(st, "setshell", "execution of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+ }
+ else if (command->meta == META_SHELL)
+ {
+ if (!runShellCommand(st, NULL, argv + 1, argc - 1))
+ {
+ commandFailed(st, "shell", "execution of meta-command failed");
+ return CSTATE_ABORTED;
+ }
+ }
+
+ /*
+ * executing the expression or shell command might have taken a
+ * non-negligible amount of time, so reset 'now'
+ */
+ INSTR_TIME_SET_ZERO(*now);
+
+ return CSTATE_END_COMMAND;
}
/*
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: pgbench - doCustom cleanup
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Re: pgbench - doCustom cleanup Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Re: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
@ 2018-11-21 18:08 ` Alvaro Herrera <[email protected]>
2 siblings, 0 replies; 25+ messages in thread
From: Alvaro Herrera @ 2018-11-21 18:08 UTC (permalink / raw)
To: Fabien COELHO <[email protected]>; +Cc: Jamison, Kirk <[email protected]>; PostgreSQL Developers <[email protected]>
On 2018-Nov-20, Fabien COELHO wrote:
> > On INSTR_TIME_SET_CURRENT_LAZY(), you cannot just put an "if" inside a
> > macro -- consider this:
> > if (foo)
> > INSTR_TIME_SET_CURRENT_LAZY(bar);
> > else
> > something_else();
> > Which "if" is the else now attached to? Now maybe the C standard has an
> > answer for that (I don't know what it is), but it's hard to read and
> > likely the compiler will complain anyway. I wrapped it in "do { }
> > while(0)" as is customary.
>
> Indeed, good catch.
Actually, reviewing this bit again, I realized that it should be a
statement that evaluates to whether the change was made or not -- this
way, it can be used in one existing place. Pushed that way. (I
verified that InstrStartNode fails in the expected way if you call it
twice.)
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v17 1/7] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 2a8683a734..710b51ff7c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21113,10 +21113,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--FCuugMFkClbJLl1L
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v18 1/6] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 64614b569c..dbfb525069 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 464a48ed6a..f26e245c65 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21122,10 +21122,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--vtzGhvizbBRQ85DL
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH 1/6] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 34bc0d0526..c299501c7b 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e04c33e4ad..6a3b1b46e7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5173,9 +5163,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 96e7fdbcfe..d1ce351200 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10324,21 +10323,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1b460a2612..21b04d5eb8 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1788,10 +1788,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2935,10 +2931,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3547,10 +3539,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ef1539044f..81d6f3819a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13501,7 +13501,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13573,7 +13577,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--mP3DRpeJDSE+ciuQ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="0002-Add-pg_depend.refobjversion-v12.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v14 1/7] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ad039e97a5..febe582d27 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13501,7 +13501,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13573,7 +13577,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--wRRV7LY7NUeQGEoC
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v14-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v16 1/7] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 28312e14ef..ce31d16bd1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13556,7 +13556,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13628,7 +13632,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--sdtB3X0nJg68CQEu
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v15 1/7] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 28312e14ef..ce31d16bd1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13556,7 +13556,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13628,7 +13632,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--bp/iNruPH9dso1Pn
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v15-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v16 1/7] Remove pg_collation.collversion.
@ 2019-05-28 18:15 Thomas Munro <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2019-05-28 18:15 UTC (permalink / raw)
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc4d7f0f78..64726779d8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
--fUYQa+Pmc3FrFX/N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0002-Add-pg_depend.refobjversion.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
@ 2025-04-04 08:52 Amit Langote <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Amit Langote @ 2025-04-04 08:52 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: David Rowley <[email protected]>; Alvaro Herrera <[email protected]>; Dmitry Dolgov <[email protected]>; [email protected]; vignesh C <[email protected]>; pgsql-hackers; Richard Guo <[email protected]>
On Fri, Apr 4, 2025 at 5:48 PM Ashutosh Bapat
<[email protected]> wrote:
> On Fri, Apr 4, 2025 at 2:04 PM Amit Langote <[email protected]> wrote:
> > I’ve now marked this as committed after pushing the patches earlier today.
>
> Thanks a lot.
Thank you too for working on it.
> > I realize the CF entry was originally about the project to reduce
> > memory usage during partitionwise join planning, but we ended up
> > committing something else. I suppose we can create a new entry if and
> > when we pick that original work back up.
>
> Will create a new CF entry just to keep the patch floated.
Sounds good.
Saving memory where we can does seem worthwhile, as long as the
approach stays simple. If there’s any doubt on that front, maybe it’s
worth spending a bit more time to see if things can be simplified.
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
@ 2026-07-01 09:15 solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: solai v @ 2026-07-01 09:15 UTC (permalink / raw)
To: Akshay Joshi <[email protected]>; +Cc: Ilmar Y <[email protected]>; [email protected]
Hi all,
On Tue, Jun 30, 2026 at 4:48 PM Akshay Joshi
<[email protected]> wrote:
>
> All,
>
> I've updated the patch to align with the interface change introduced by commit d6ed87d1989 (Andrew Dunstan, 2026-06-26 "Use named boolean parameters for pg_get_*_ddl option arguments").
>
> That commit replaced the VARIADIC text[] alternating key/value option interface with typed named boolean parameters across pg_get_role_ddl(), pg_get_tablespace_ddl(), and pg_get_database_ddl(), removing the DdlOption/parse_ddl_options() machinery in favour of direct PG_GETARG_BOOL() calls.
>
> Updated patch v14 is ready for review/commit.
>
I reviewed and tested the v14 patch on the latest master. The patch
applied cleanly, built successfully, and passed make check without any
regression failures. I verified the new function signature and
confirmed that pg_get_policy_ddl() now uses the new interface with the
boolean DEFAULT false argument. Also I tested the function with a
variety of row-level security policies, including: Basic policy
reconstruction, PERMISSIVE and RESTRICTIVE policies, Different command
types (SELECT, INSERT, UPDATE, DELETE), Policies with multiple roles
and the PUBLIC role, Quoted table, policy, and role names, USING and
WITH CHECK clauses, Complex expressions including EXISTS, nested
subqueries, CASE expressions, COALESCE, ANY/ALL operators, and boolean
expressions, NULL inputs, invalid relation names, and non-existent
policies. The generated DDL was correct as per the intent, and the
expected errors were verified for the invalid inputs. I also verified
that the generated DDL is executable by dropping an existing policy,
recreating it using the output of pg_get_policy_ddl(), and confirming
that the recreated policy was reconstructed successfully again by the
function.
One observation I noted is that for policies using default attributes
(TO PUBLIC and AS PERMISSIVE), the generated DDL omits those clauses
and produces a semantically equivalent statement which seems
intentional but thought of sending it here for further clarification.
Overall, I did not encounter any functional issues during testing. The
patch looks good to me and worked as expected in all the scenarios I
tested.
Regards,
Solai
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
@ 2026-07-01 10:31 ` Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Akshay Joshi @ 2026-07-01 10:31 UTC (permalink / raw)
To: solai v <[email protected]>; +Cc: Ilmar Y <[email protected]>; [email protected]
On Wed, Jul 1, 2026 at 2:44 PM solai v <[email protected]> wrote:
> Hi all,
>
>
> On Tue, Jun 30, 2026 at 4:48 PM Akshay Joshi
> <[email protected]> wrote:
> >
> > All,
> >
> > I've updated the patch to align with the interface change introduced by
> commit d6ed87d1989 (Andrew Dunstan, 2026-06-26 "Use named boolean
> parameters for pg_get_*_ddl option arguments").
> >
> > That commit replaced the VARIADIC text[] alternating key/value option
> interface with typed named boolean parameters across pg_get_role_ddl(),
> pg_get_tablespace_ddl(), and pg_get_database_ddl(), removing the
> DdlOption/parse_ddl_options() machinery in favour of direct
> PG_GETARG_BOOL() calls.
> >
> > Updated patch v14 is ready for review/commit.
> >
>
>
> I reviewed and tested the v14 patch on the latest master. The patch
> applied cleanly, built successfully, and passed make check without any
> regression failures. I verified the new function signature and
> confirmed that pg_get_policy_ddl() now uses the new interface with the
> boolean DEFAULT false argument. Also I tested the function with a
> variety of row-level security policies, including: Basic policy
> reconstruction, PERMISSIVE and RESTRICTIVE policies, Different command
> types (SELECT, INSERT, UPDATE, DELETE), Policies with multiple roles
> and the PUBLIC role, Quoted table, policy, and role names, USING and
> WITH CHECK clauses, Complex expressions including EXISTS, nested
> subqueries, CASE expressions, COALESCE, ANY/ALL operators, and boolean
> expressions, NULL inputs, invalid relation names, and non-existent
> policies. The generated DDL was correct as per the intent, and the
> expected errors were verified for the invalid inputs. I also verified
> that the generated DDL is executable by dropping an existing policy,
> recreating it using the output of pg_get_policy_ddl(), and confirming
> that the recreated policy was reconstructed successfully again by the
> function.
> One observation I noted is that for policies using default attributes
> (TO PUBLIC and AS PERMISSIVE), the generated DDL omits those clauses
> and produces a semantically equivalent statement which seems
> intentional but thought of sending it here for further clarification.
> Overall, I did not encounter any functional issues during testing. The
> patch looks good to me and worked as expected in all the scenarios I
> tested.
>
Yes, it's intentional for all pg_get_***_ddl functions. Clause with
defaults won't be reconstructed.
>
>
> Regards,
> Solai
>
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
@ 2026-07-01 11:31 ` solai v <[email protected]>
2026-07-06 11:28 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: solai v @ 2026-07-01 11:31 UTC (permalink / raw)
To: Akshay Joshi <[email protected]>; +Cc: Ilmar Y <[email protected]>; [email protected]
Hi all,
On Wed, Jul 1, 2026 at 4:01 PM Akshay Joshi
<[email protected]> wrote:
>
>
>
> On Wed, Jul 1, 2026 at 2:44 PM solai v <[email protected]> wrote:
>>
>> Hi all,
>>
>>
>> On Tue, Jun 30, 2026 at 4:48 PM Akshay Joshi
>> <[email protected]> wrote:
>> >
>> > All,
>> >
>> > I've updated the patch to align with the interface change introduced by commit d6ed87d1989 (Andrew Dunstan, 2026-06-26 "Use named boolean parameters for pg_get_*_ddl option arguments").
>> >
>> > That commit replaced the VARIADIC text[] alternating key/value option interface with typed named boolean parameters across pg_get_role_ddl(), pg_get_tablespace_ddl(), and pg_get_database_ddl(), removing the DdlOption/parse_ddl_options() machinery in favour of direct PG_GETARG_BOOL() calls.
>> >
>> > Updated patch v14 is ready for review/commit.
>> >
>>
>>
>> I reviewed and tested the v14 patch on the latest master. The patch
>> applied cleanly, built successfully, and passed make check without any
>> regression failures. I verified the new function signature and
>> confirmed that pg_get_policy_ddl() now uses the new interface with the
>> boolean DEFAULT false argument. Also I tested the function with a
>> variety of row-level security policies, including: Basic policy
>> reconstruction, PERMISSIVE and RESTRICTIVE policies, Different command
>> types (SELECT, INSERT, UPDATE, DELETE), Policies with multiple roles
>> and the PUBLIC role, Quoted table, policy, and role names, USING and
>> WITH CHECK clauses, Complex expressions including EXISTS, nested
>> subqueries, CASE expressions, COALESCE, ANY/ALL operators, and boolean
>> expressions, NULL inputs, invalid relation names, and non-existent
>> policies. The generated DDL was correct as per the intent, and the
>> expected errors were verified for the invalid inputs. I also verified
>> that the generated DDL is executable by dropping an existing policy,
>> recreating it using the output of pg_get_policy_ddl(), and confirming
>> that the recreated policy was reconstructed successfully again by the
>> function.
>> One observation I noted is that for policies using default attributes
>> (TO PUBLIC and AS PERMISSIVE), the generated DDL omits those clauses
>> and produces a semantically equivalent statement which seems
>> intentional but thought of sending it here for further clarification.
>> Overall, I did not encounter any functional issues during testing. The
>> patch looks good to me and worked as expected in all the scenarios I
>> tested.
>
>
> Yes, it's intentional for all pg_get_***_ddl functions. Clause with defaults won't be reconstructed.
>>
>>
Ok. Thank you for the clarification.
Regards,
Solai
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
@ 2026-07-06 11:28 ` Akshay Joshi <[email protected]>
2026-07-08 16:58 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Akshay Joshi @ 2026-07-06 11:28 UTC (permalink / raw)
To: solai v <[email protected]>; +Cc: Ilmar Y <[email protected]>; [email protected]
All,
I found that OID 6517 was already taken by the max(uuid) aggregate. I've
reassigned pg_get_policy_ddl to OID 9683.
The v15 patch is ready for review and commit.
On Wed, Jul 1, 2026 at 5:01 PM solai v <[email protected]> wrote:
> Hi all,
>
> On Wed, Jul 1, 2026 at 4:01 PM Akshay Joshi
> <[email protected]> wrote:
> >
> >
> >
> > On Wed, Jul 1, 2026 at 2:44 PM solai v <[email protected]> wrote:
> >>
> >> Hi all,
> >>
> >>
> >> On Tue, Jun 30, 2026 at 4:48 PM Akshay Joshi
> >> <[email protected]> wrote:
> >> >
> >> > All,
> >> >
> >> > I've updated the patch to align with the interface change introduced
> by commit d6ed87d1989 (Andrew Dunstan, 2026-06-26 "Use named boolean
> parameters for pg_get_*_ddl option arguments").
> >> >
> >> > That commit replaced the VARIADIC text[] alternating key/value option
> interface with typed named boolean parameters across pg_get_role_ddl(),
> pg_get_tablespace_ddl(), and pg_get_database_ddl(), removing the
> DdlOption/parse_ddl_options() machinery in favour of direct
> PG_GETARG_BOOL() calls.
> >> >
> >> > Updated patch v14 is ready for review/commit.
> >> >
> >>
> >>
> >> I reviewed and tested the v14 patch on the latest master. The patch
> >> applied cleanly, built successfully, and passed make check without any
> >> regression failures. I verified the new function signature and
> >> confirmed that pg_get_policy_ddl() now uses the new interface with the
> >> boolean DEFAULT false argument. Also I tested the function with a
> >> variety of row-level security policies, including: Basic policy
> >> reconstruction, PERMISSIVE and RESTRICTIVE policies, Different command
> >> types (SELECT, INSERT, UPDATE, DELETE), Policies with multiple roles
> >> and the PUBLIC role, Quoted table, policy, and role names, USING and
> >> WITH CHECK clauses, Complex expressions including EXISTS, nested
> >> subqueries, CASE expressions, COALESCE, ANY/ALL operators, and boolean
> >> expressions, NULL inputs, invalid relation names, and non-existent
> >> policies. The generated DDL was correct as per the intent, and the
> >> expected errors were verified for the invalid inputs. I also verified
> >> that the generated DDL is executable by dropping an existing policy,
> >> recreating it using the output of pg_get_policy_ddl(), and confirming
> >> that the recreated policy was reconstructed successfully again by the
> >> function.
> >> One observation I noted is that for policies using default attributes
> >> (TO PUBLIC and AS PERMISSIVE), the generated DDL omits those clauses
> >> and produces a semantically equivalent statement which seems
> >> intentional but thought of sending it here for further clarification.
> >> Overall, I did not encounter any functional issues during testing. The
> >> patch looks good to me and worked as expected in all the scenarios I
> >> tested.
> >
> >
> > Yes, it's intentional for all pg_get_***_ddl functions. Clause with
> defaults won't be reconstructed.
> >>
> >>
>
> Ok. Thank you for the clarification.
>
> Regards,
> Solai
>
Attachments:
[application/octet-stream] v15-0001-Add-pg_get_policy_ddl-to-reconstruct-CREATE.patch (31.5K, ../../CANxoLDfUeoh+X1AeazXAQjYL312nTa6bY+=iPNJjmN4bXwSKWw@mail.gmail.com/3-v15-0001-Add-pg_get_policy_ddl-to-reconstruct-CREATE.patch)
download | inline diff:
From 30aa1e5001737ff0ce675aa8adf8ae7e6cda633d Mon Sep 17 00:00:00 2001
From: Akshay Joshi <[email protected]>
Date: Fri, 22 May 2026 18:18:07 +0530
Subject: [PATCH v15] Add pg_get_policy_ddl() to reconstruct CREATE POLICY
statements
pg_get_policy_ddl(table regclass,
policyname name,
pretty bool DEFAULT false)
RETURNS SETOF text
reconstructs the CREATE POLICY statement for the named row-level
security policy on the specified table. Although a single statement is
produced, the function returns a SETOF text result so its calling
convention matches the rest of the pg_get_*_ddl family.
The reconstructed DDL includes all clauses of the CREATE POLICY syntax:
the policy's permissiveness (AS RESTRICTIVE), command type (FOR
SELECT/INSERT/UPDATE/DELETE), role list (TO <roles>), USING
qualification, and WITH CHECK expression. Clauses whose value equals
the parser's default -- PERMISSIVE, FOR ALL, and TO PUBLIC -- are
omitted from the output, matching the convention used by pg_get_indexdef,
pg_get_constraintdef, pg_get_viewdef, and similar functions. The result
is therefore semantically equivalent to the original DDL, not lexically
identical.
The pretty parameter controls output formatting and defaults to false,
following the same convention as pg_get_role_ddl, pg_get_tablespace_ddl,
and pg_get_database_ddl. NULL passed explicitly for pretty is treated
as false.
NULL inputs for the table or policy name yield no rows. An invalid
relation name surfaces the regclass resolution error; a non-existent
policy raises an explicit "policy ... does not exist" error.
Usage examples:
-- compact form (default)
SELECT * FROM pg_get_policy_ddl('rls_table', 'pol1');
SELECT * FROM pg_get_policy_ddl(16564, 'pol1');
-- pretty-printed form
SELECT * FROM pg_get_policy_ddl('rls_table', 'pol1', true);
Regression coverage is added to src/test/regress/sql/rowsecurity.sql
and exercises all valid combinations of the CREATE POLICY syntax:
PERMISSIVE/RESTRICTIVE, all FOR command variants (ALL/SELECT/INSERT/
UPDATE/DELETE), multi-role TO lists, USING-only, WITH CHECK-only, and
USING+WITH CHECK policies for both ALL and UPDATE commands (the only
two that accept both), RESTRICTIVE on a specific command type,
subquery expressions, pretty and non-pretty output, all boolean
representations for the pretty argument (true/false, on/off, 1/0),
NULL and error paths, and a round-trip test that drops and re-executes
the generated DDL.
Author: Akshay Joshi <[email protected]>
---
doc/src/sgml/func/func-info.sgml | 19 ++
src/backend/utils/adt/ddlutils.c | 259 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +
src/test/regress/expected/rowsecurity.out | 291 ++++++++++++++++++++++
src/test/regress/sql/rowsecurity.sql | 128 ++++++++++
5 files changed, 703 insertions(+)
diff --git a/doc/src/sgml/func/func-info.sgml b/doc/src/sgml/func/func-info.sgml
index 69ef3857cfa..858ea4609a6 100644
--- a/doc/src/sgml/func/func-info.sgml
+++ b/doc/src/sgml/func/func-info.sgml
@@ -3960,6 +3960,25 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
is false, the <literal>OWNER</literal> clause is omitted.
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_policy_ddl</primary>
+ </indexterm>
+ <function>pg_get_policy_ddl</function>
+ ( <parameter>table</parameter> <type>regclass</type>,
+ <parameter>policy_name</parameter> <type>name</type>
+ <optional>, <parameter>pretty</parameter> <type>boolean</type>
+ </optional> )
+ <returnvalue>setof text</returnvalue>
+ </para>
+ <para>
+ Reconstructs the <command>CREATE POLICY</command> statement for the
+ named row-level security policy on the specified table. The result
+ is returned as a single row. If <parameter>pretty</parameter> is
+ true, the output is formatted for readability; the default is false.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/src/backend/utils/adt/ddlutils.c b/src/backend/utils/adt/ddlutils.c
index a70f1c28655..cf7b596085f 100644
--- a/src/backend/utils/adt/ddlutils.c
+++ b/src/backend/utils/adt/ddlutils.c
@@ -26,6 +26,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_database.h"
#include "catalog/pg_db_role_setting.h"
+#include "catalog/pg_policy.h"
#include "catalog/pg_tablespace.h"
#include "commands/tablespace.h"
#include "common/relpath.h"
@@ -56,6 +57,9 @@ static List *pg_get_tablespace_ddl_internal(Oid tsid, bool pretty, bool no_owner
static Datum pg_get_tablespace_ddl_srf(FunctionCallInfo fcinfo, Oid tsid);
static List *pg_get_database_ddl_internal(Oid dbid, bool pretty,
bool no_owner, bool no_tablespace);
+static List *pg_get_policy_ddl_internal(Oid tableID, const char *policyName,
+ bool pretty);
+static const char *get_policy_cmd_name(char cmd);
/*
@@ -975,3 +979,258 @@ pg_get_database_ddl(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
}
+
+/*
+ * get_policy_cmd_name
+ * Map a non-ALL pg_policy.polcmd char to its SQL keyword.
+ *
+ * Callers must guard against '*' (ALL) themselves; it is not passed here.
+ */
+static const char *
+get_policy_cmd_name(char cmd)
+{
+ switch (cmd)
+ {
+ case ACL_SELECT_CHR:
+ return "SELECT";
+ case ACL_INSERT_CHR:
+ return "INSERT";
+ case ACL_UPDATE_CHR:
+ return "UPDATE";
+ case ACL_DELETE_CHR:
+ return "DELETE";
+ default:
+ elog(ERROR, "unrecognized policy command: %d", (int) cmd);
+ return NULL; /* keep compiler quiet */
+ }
+}
+
+/*
+ * pg_get_policy_ddl_internal
+ * Generate the DDL statement to recreate a row-level security policy.
+ *
+ * Returns a List containing a single palloc'd string with the CREATE POLICY
+ * statement. Returning a List keeps the calling convention consistent with
+ * the rest of the pg_get_*_ddl family even though only one row is produced.
+ */
+static List *
+pg_get_policy_ddl_internal(Oid tableID, const char *policyName, bool pretty)
+{
+ Relation pgPolicyRel;
+ HeapTuple tuplePolicy;
+ Form_pg_policy policyForm;
+ ScanKeyData skey[2];
+ SysScanDesc sscan;
+ StringInfoData buf;
+ Datum valueDatum;
+ bool attrIsNull;
+ char *relname;
+ char *nspname;
+ char *targetTable;
+ List *statements = NIL;
+
+ /* Validate that the relation exists and build its qualified name. */
+ relname = get_rel_name(tableID);
+ if (relname == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("relation with OID %u does not exist", tableID)));
+
+ nspname = get_namespace_name(get_rel_namespace(tableID));
+ if (nspname == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_SCHEMA),
+ errmsg("schema for relation with OID %u does not exist",
+ tableID)));
+
+ targetTable = quote_qualified_identifier(nspname, relname);
+ pfree(relname);
+ pfree(nspname);
+
+ pgPolicyRel = table_open(PolicyRelationId, AccessShareLock);
+
+ /* Set key - policy's relation id. */
+ ScanKeyInit(&skey[0],
+ Anum_pg_policy_polrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tableID));
+
+ /* Set key - policy's name. */
+ ScanKeyInit(&skey[1],
+ Anum_pg_policy_polname,
+ BTEqualStrategyNumber, F_NAMEEQ,
+ CStringGetDatum(policyName));
+
+ sscan = systable_beginscan(pgPolicyRel,
+ PolicyPolrelidPolnameIndexId, true, NULL, 2,
+ skey);
+
+ tuplePolicy = systable_getnext(sscan);
+ if (!HeapTupleIsValid(tuplePolicy))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("policy \"%s\" for table \"%s\" does not exist",
+ policyName, targetTable)));
+
+ policyForm = (Form_pg_policy) GETSTRUCT(tuplePolicy);
+
+ initStringInfo(&buf);
+
+ /* Build the CREATE POLICY statement */
+ appendStringInfo(&buf, "CREATE POLICY %s ON %s",
+ quote_identifier(policyName),
+ targetTable);
+ pfree(targetTable);
+
+ /*
+ * Emit AS RESTRICTIVE only when it differs from the default (PERMISSIVE).
+ */
+ if (!policyForm->polpermissive)
+ append_ddl_option(&buf, pretty, 4, "AS RESTRICTIVE");
+
+ /*
+ * Emit FOR <cmd> only when it differs from the default (ALL, encoded as
+ * '*').
+ */
+ if (policyForm->polcmd != '*')
+ append_ddl_option(&buf, pretty, 4, "FOR %s",
+ get_policy_cmd_name(policyForm->polcmd));
+
+ /*
+ * Emit TO <roles> only when it differs from the default (PUBLIC). PUBLIC
+ * is encoded in polroles as a single InvalidOid element, so we omit the
+ * clause whenever every entry is InvalidOid. polroles is always non-NULL.
+ */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polroles,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ Assert(!attrIsNull);
+ {
+ ArrayType *policy_roles = DatumGetArrayTypeP(valueDatum);
+ int nitems = ARR_DIMS(policy_roles)[0];
+ Oid *roles = (Oid *) ARR_DATA_PTR(policy_roles);
+ StringInfoData role_names;
+
+ initStringInfo(&role_names);
+
+ for (int i = 0; i < nitems; i++)
+ {
+ if (OidIsValid(roles[i]))
+ {
+ char *rolename = GetUserNameFromId(roles[i], false);
+
+ if (role_names.len > 0)
+ appendStringInfoString(&role_names, ", ");
+ appendStringInfoString(&role_names, quote_identifier(rolename));
+ pfree(rolename);
+ }
+ }
+
+ if (role_names.len > 0)
+ append_ddl_option(&buf, pretty, 4, "TO %s", role_names.data);
+
+ pfree(role_names.data);
+ }
+
+ /* USING expression */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polqual,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ if (!attrIsNull)
+ {
+ Datum expr;
+
+ expr = DirectFunctionCall3(pg_get_expr_ext,
+ valueDatum,
+ ObjectIdGetDatum(policyForm->polrelid),
+ BoolGetDatum(pretty));
+ append_ddl_option(&buf, pretty, 4, "USING (%s)",
+ TextDatumGetCString(expr));
+ }
+
+ /* WITH CHECK expression */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polwithcheck,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ if (!attrIsNull)
+ {
+ Datum expr;
+
+ expr = DirectFunctionCall3(pg_get_expr_ext,
+ valueDatum,
+ ObjectIdGetDatum(policyForm->polrelid),
+ BoolGetDatum(pretty));
+ append_ddl_option(&buf, pretty, 4, "WITH CHECK (%s)",
+ TextDatumGetCString(expr));
+ }
+
+ appendStringInfoChar(&buf, ';');
+
+ statements = lappend(statements, pstrdup(buf.data));
+
+ systable_endscan(sscan);
+ table_close(pgPolicyRel, AccessShareLock);
+ pfree(buf.data);
+
+ return statements;
+}
+
+/*
+ * pg_get_policy_ddl
+ * Return DDL to recreate a row-level security policy as a single text row.
+ */
+Datum
+pg_get_policy_ddl(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ List *statements;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ MemoryContext oldcontext;
+ Oid tableID;
+ Name policyName;
+ bool pretty;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+ if (PG_ARGISNULL(0) || PG_ARGISNULL(1))
+ {
+ MemoryContextSwitchTo(oldcontext);
+ SRF_RETURN_DONE(funcctx);
+ }
+
+ tableID = PG_GETARG_OID(0);
+ policyName = PG_GETARG_NAME(1);
+ pretty = !PG_ARGISNULL(2) && PG_GETARG_BOOL(2);
+
+ statements = pg_get_policy_ddl_internal(tableID,
+ NameStr(*policyName),
+ pretty);
+ funcctx->user_fctx = statements;
+ funcctx->max_calls = list_length(statements);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
+ statements = (List *) funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ char *stmt;
+
+ stmt = list_nth(statements, funcctx->call_cntr);
+
+ SRF_RETURN_NEXT(funcctx, CStringGetTextDatum(stmt));
+ }
+ else
+ {
+ list_free_deep(statements);
+ SRF_RETURN_DONE(funcctx);
+ }
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..cd029b483f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8634,6 +8634,12 @@
proargtypes => 'regdatabase bool bool bool',
proargnames => '{database,pretty,owner,tablespace}',
proargdefaults => '{false,true,true}', prosrc => 'pg_get_database_ddl' },
+{ oid => '9683', descr => 'get DDL to recreate a row-level security policy',
+ proname => 'pg_get_policy_ddl', prorows => '1', proisstrict => 'f',
+ proretset => 't', provolatile => 's', pronargdefaults => '1',
+ prorettype => 'text', proargtypes => 'regclass name bool',
+ proargnames => '{table,policyname,pretty}', proargdefaults => '{false}',
+ prosrc => 'pg_get_policy_ddl' },
{ oid => '2509',
descr => 'deparse an encoded expression with pretty-print option',
proname => 'pg_get_expr', provolatile => 's', prorettype => 'text',
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 3a5e82c35bd..bf54f66339f 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -5195,6 +5195,297 @@ reset rls_test.blah;
drop function rls_f(text);
drop table rls_t, test_t;
--
+-- Test for pg_get_policy_ddl(table, policy_name[, pretty]) function.
+--
+CREATE TABLE rls_tbl_1 (
+ did int primary key,
+ cid int,
+ dlevel int not null,
+ dauthor name,
+ dtitle text
+);
+GRANT ALL ON rls_tbl_1 TO public;
+CREATE TABLE rls_tbl_2 (
+ pguser name primary key,
+ seclv int
+);
+GRANT SELECT ON rls_tbl_2 TO public;
+-- Test PERMISSIVE and RESTRICTIVE
+CREATE POLICY rls_p1 ON rls_tbl_1 AS PERMISSIVE
+ USING (dlevel <= (SELECT seclv FROM rls_tbl_2 WHERE pguser = current_user));
+CREATE POLICY rls_p2 ON rls_tbl_1 AS RESTRICTIVE USING (cid <> 44 AND cid < 50);
+-- Test FOR ALL | SELECT | INSERT | UPDATE | DELETE
+CREATE POLICY rls_p3 ON rls_tbl_1 FOR ALL USING (dauthor = current_user);
+CREATE POLICY rls_p4 ON rls_tbl_1 FOR SELECT USING (cid % 2 = 0);
+CREATE POLICY rls_p5 ON rls_tbl_1 FOR INSERT WITH CHECK (cid % 2 = 1);
+CREATE POLICY rls_p6 ON rls_tbl_1 FOR UPDATE USING (cid % 2 = 0);
+CREATE POLICY rls_p7 ON rls_tbl_1 FOR DELETE USING (cid < 8);
+-- Test TO { role_name ... }
+CREATE POLICY rls_p8 ON rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+CREATE POLICY rls_p9 ON rls_tbl_1 TO regress_rls_exempt_user WITH CHECK (cid = (SELECT seclv FROM rls_tbl_2));
+-- Test UPDATE policy with both USING and WITH CHECK
+CREATE POLICY rls_p10 ON rls_tbl_1 FOR UPDATE
+ USING (dlevel > 0) WITH CHECK (dlevel < 100);
+-- Test ALL policy with both USING and WITH CHECK (AS PERMISSIVE and FOR ALL are defaults, omitted)
+CREATE POLICY rls_p11 ON rls_tbl_1
+ USING (dlevel >= 1) WITH CHECK (dlevel <= 99);
+-- Test RESTRICTIVE on a specific command
+CREATE POLICY rls_p12 ON rls_tbl_1 AS RESTRICTIVE FOR SELECT
+ USING (cid > 0);
+-- NULL inputs should return no rows
+SELECT count(*) FROM pg_get_policy_ddl(NULL, 'rls_p1');
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM pg_get_policy_ddl('rls_tbl_1', NULL);
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM pg_get_policy_ddl(NULL, NULL);
+ count
+-------
+ 0
+(1 row)
+
+-- Table does not exist
+SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1');
+ERROR: relation "nonexistent_tbl" does not exist
+LINE 1: SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1')...
+ ^
+-- Policy does not exist
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'nonexistent_pol');
+ERROR: policy "nonexistent_pol" for table "regress_rls_schema.rls_tbl_1" does not exist
+-- Without pretty formatting (default); also verify explicit false equals default
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1');
+ pg_get_policy_ddl
+-------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p1 ON regress_rls_schema.rls_tbl_1 USING ((dlevel <= ( SELECT rls_tbl_2.seclv+
+ FROM rls_tbl_2 +
+ WHERE (rls_tbl_2.pguser = CURRENT_USER))));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p2 ON regress_rls_schema.rls_tbl_1 AS RESTRICTIVE USING (((cid <> 44) AND (cid < 50)));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------
+ CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p4 ON regress_rls_schema.rls_tbl_1 FOR SELECT USING (((cid % 2) = 0));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p5 ON regress_rls_schema.rls_tbl_1 FOR INSERT WITH CHECK (((cid % 2) = 1));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p6 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING (((cid % 2) = 0));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------
+ CREATE POLICY rls_p7 ON regress_rls_schema.rls_tbl_1 FOR DELETE USING ((cid < 8));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p8 ON regress_rls_schema.rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p9 ON regress_rls_schema.rls_tbl_1 TO regress_rls_exempt_user WITH CHECK ((cid = ( SELECT rls_tbl_2.seclv+
+ FROM rls_tbl_2)));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10');
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p11 ON regress_rls_schema.rls_tbl_1 USING ((dlevel >= 1)) WITH CHECK ((dlevel <= 99));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p12 ON regress_rls_schema.rls_tbl_1 AS RESTRICTIVE FOR SELECT USING ((cid > 0));
+(1 row)
+
+-- Explicit false must match omitting the argument
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', false);
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+-- NULL pretty must also default to non-pretty
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', NULL);
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+-- pretty accepts all standard boolean representations: true/false, on/off, 1/0
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'on'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '1'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'off'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '0'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+\pset format aligned
+-- With pretty formatting
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p1 ON regress_rls_schema.rls_tbl_1
+ USING (dlevel <= (( SELECT rls_tbl_2.seclv
+ FROM rls_tbl_2
+ WHERE rls_tbl_2.pguser = CURRENT_USER)));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p2 ON regress_rls_schema.rls_tbl_1
+ AS RESTRICTIVE
+ USING (cid <> 44 AND cid < 50);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p4 ON regress_rls_schema.rls_tbl_1
+ FOR SELECT
+ USING ((cid % 2) = 0);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p5 ON regress_rls_schema.rls_tbl_1
+ FOR INSERT
+ WITH CHECK ((cid % 2) = 1);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p6 ON regress_rls_schema.rls_tbl_1
+ FOR UPDATE
+ USING ((cid % 2) = 0);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p7 ON regress_rls_schema.rls_tbl_1
+ FOR DELETE
+ USING (cid < 8);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p8 ON regress_rls_schema.rls_tbl_1
+ TO regress_rls_dave, regress_rls_alice
+ USING (true);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p9 ON regress_rls_schema.rls_tbl_1
+ TO regress_rls_exempt_user
+ WITH CHECK (cid = (( SELECT rls_tbl_2.seclv
+ FROM rls_tbl_2)));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1
+ FOR UPDATE
+ USING (dlevel > 0)
+ WITH CHECK (dlevel < 100);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p11 ON regress_rls_schema.rls_tbl_1
+ USING (dlevel >= 1)
+ WITH CHECK (dlevel <= 99);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p12 ON regress_rls_schema.rls_tbl_1
+ AS RESTRICTIVE
+ FOR SELECT
+ USING (cid > 0);
+(1 row)
+\pset format aligned
+-- Round-trip: the generated DDL must be re-executable, including for atomic
+-- boolean expressions that pg_get_expr() does not parenthesize.
+CREATE TABLE rls_rt (a int);
+CREATE POLICY rt_true ON rls_rt USING (true);
+CREATE POLICY rt_false ON rls_rt FOR INSERT WITH CHECK (false);
+CREATE TEMP TABLE rt_ddl AS
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_true') AS ddl
+ UNION ALL
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_false');
+DROP POLICY rt_true ON rls_rt;
+DROP POLICY rt_false ON rls_rt;
+SELECT ddl FROM rt_ddl ORDER BY ddl \gexec
+CREATE POLICY rt_false ON regress_rls_schema.rls_rt FOR INSERT WITH CHECK (false);
+CREATE POLICY rt_true ON regress_rls_schema.rls_rt USING (true);
+SELECT polname FROM pg_policy WHERE polrelid = 'rls_rt'::regclass ORDER BY polname;
+ polname
+----------
+ rt_false
+ rt_true
+(2 rows)
+
+DROP TABLE rls_rt;
+-- Clean up objects created for testing pg_get_policy_ddl function.
+DROP POLICY rls_p1 ON rls_tbl_1;
+DROP POLICY rls_p2 ON rls_tbl_1;
+DROP POLICY rls_p3 ON rls_tbl_1;
+DROP POLICY rls_p4 ON rls_tbl_1;
+DROP POLICY rls_p5 ON rls_tbl_1;
+DROP POLICY rls_p6 ON rls_tbl_1;
+DROP POLICY rls_p7 ON rls_tbl_1;
+DROP POLICY rls_p8 ON rls_tbl_1;
+DROP POLICY rls_p9 ON rls_tbl_1;
+DROP POLICY rls_p10 ON rls_tbl_1;
+DROP POLICY rls_p11 ON rls_tbl_1;
+DROP POLICY rls_p12 ON rls_tbl_1;
+DROP TABLE rls_tbl_1;
+DROP TABLE rls_tbl_2;
+--
-- Clean up objects
--
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/rowsecurity.sql b/src/test/regress/sql/rowsecurity.sql
index 6b3566271df..e4d64813e65 100644
--- a/src/test/regress/sql/rowsecurity.sql
+++ b/src/test/regress/sql/rowsecurity.sql
@@ -2598,6 +2598,134 @@ reset rls_test.blah;
drop function rls_f(text);
drop table rls_t, test_t;
+--
+-- Test for pg_get_policy_ddl(table, policy_name[, pretty]) function.
+--
+CREATE TABLE rls_tbl_1 (
+ did int primary key,
+ cid int,
+ dlevel int not null,
+ dauthor name,
+ dtitle text
+);
+GRANT ALL ON rls_tbl_1 TO public;
+CREATE TABLE rls_tbl_2 (
+ pguser name primary key,
+ seclv int
+);
+GRANT SELECT ON rls_tbl_2 TO public;
+
+-- Test PERMISSIVE and RESTRICTIVE
+CREATE POLICY rls_p1 ON rls_tbl_1 AS PERMISSIVE
+ USING (dlevel <= (SELECT seclv FROM rls_tbl_2 WHERE pguser = current_user));
+CREATE POLICY rls_p2 ON rls_tbl_1 AS RESTRICTIVE USING (cid <> 44 AND cid < 50);
+
+-- Test FOR ALL | SELECT | INSERT | UPDATE | DELETE
+CREATE POLICY rls_p3 ON rls_tbl_1 FOR ALL USING (dauthor = current_user);
+CREATE POLICY rls_p4 ON rls_tbl_1 FOR SELECT USING (cid % 2 = 0);
+CREATE POLICY rls_p5 ON rls_tbl_1 FOR INSERT WITH CHECK (cid % 2 = 1);
+CREATE POLICY rls_p6 ON rls_tbl_1 FOR UPDATE USING (cid % 2 = 0);
+CREATE POLICY rls_p7 ON rls_tbl_1 FOR DELETE USING (cid < 8);
+
+-- Test TO { role_name ... }
+CREATE POLICY rls_p8 ON rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+CREATE POLICY rls_p9 ON rls_tbl_1 TO regress_rls_exempt_user WITH CHECK (cid = (SELECT seclv FROM rls_tbl_2));
+
+-- Test UPDATE policy with both USING and WITH CHECK
+CREATE POLICY rls_p10 ON rls_tbl_1 FOR UPDATE
+ USING (dlevel > 0) WITH CHECK (dlevel < 100);
+
+-- Test ALL policy with both USING and WITH CHECK (AS PERMISSIVE and FOR ALL are defaults, omitted)
+CREATE POLICY rls_p11 ON rls_tbl_1
+ USING (dlevel >= 1) WITH CHECK (dlevel <= 99);
+
+-- Test RESTRICTIVE on a specific command
+CREATE POLICY rls_p12 ON rls_tbl_1 AS RESTRICTIVE FOR SELECT
+ USING (cid > 0);
+
+-- NULL inputs should return no rows
+SELECT count(*) FROM pg_get_policy_ddl(NULL, 'rls_p1');
+SELECT count(*) FROM pg_get_policy_ddl('rls_tbl_1', NULL);
+SELECT count(*) FROM pg_get_policy_ddl(NULL, NULL);
+
+-- Table does not exist
+SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1');
+-- Policy does not exist
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'nonexistent_pol');
+
+-- Without pretty formatting (default); also verify explicit false equals default
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12');
+-- Explicit false must match omitting the argument
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', false);
+-- NULL pretty must also default to non-pretty
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', NULL);
+
+-- pretty accepts all standard boolean representations: true/false, on/off, 1/0
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'on'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '1'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'off'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '0'::bool);
+\pset format aligned
+
+-- With pretty formatting
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12', true);
+\pset format aligned
+
+-- Round-trip: the generated DDL must be re-executable, including for atomic
+-- boolean expressions that pg_get_expr() does not parenthesize.
+CREATE TABLE rls_rt (a int);
+CREATE POLICY rt_true ON rls_rt USING (true);
+CREATE POLICY rt_false ON rls_rt FOR INSERT WITH CHECK (false);
+CREATE TEMP TABLE rt_ddl AS
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_true') AS ddl
+ UNION ALL
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_false');
+DROP POLICY rt_true ON rls_rt;
+DROP POLICY rt_false ON rls_rt;
+SELECT ddl FROM rt_ddl ORDER BY ddl \gexec
+SELECT polname FROM pg_policy WHERE polrelid = 'rls_rt'::regclass ORDER BY polname;
+DROP TABLE rls_rt;
+
+-- Clean up objects created for testing pg_get_policy_ddl function.
+DROP POLICY rls_p1 ON rls_tbl_1;
+DROP POLICY rls_p2 ON rls_tbl_1;
+DROP POLICY rls_p3 ON rls_tbl_1;
+DROP POLICY rls_p4 ON rls_tbl_1;
+DROP POLICY rls_p5 ON rls_tbl_1;
+DROP POLICY rls_p6 ON rls_tbl_1;
+DROP POLICY rls_p7 ON rls_tbl_1;
+DROP POLICY rls_p8 ON rls_tbl_1;
+DROP POLICY rls_p9 ON rls_tbl_1;
+DROP POLICY rls_p10 ON rls_tbl_1;
+DROP POLICY rls_p11 ON rls_tbl_1;
+DROP POLICY rls_p12 ON rls_tbl_1;
+DROP TABLE rls_tbl_1;
+DROP TABLE rls_tbl_2;
+
--
-- Clean up objects
--
--
2.51.0
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-06 11:28 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
@ 2026-07-08 16:58 ` Rui Zhao <[email protected]>
2026-07-09 10:57 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Rui Zhao @ 2026-07-08 16:58 UTC (permalink / raw)
To: Akshay Joshi <[email protected]>; +Cc: solai v <[email protected]>; Ilmar Y <[email protected]>; [email protected]
Hi Akshay,
I tested v15 on current master (57f93af36f): builds, make check passes, and
basic reconstruction, pretty mode, NULL handling, and default-clause omission
all look good. Two things.
1) Back in the v3/v4 discussion you decided the ON <table> name should always
be schema-qualified for safety (the pg_get_triggerdef_worker thread with
jian he), and that pretty here only controls formatting, not schema (Phil's
point). That reasoning didn't reach the USING / WITH CHECK expressions,
though: object references inside them are still qualified only by the caller's
search_path, so they can lose their schema and rebind to a different object
when the DDL is replayed elsewhere:
CREATE FUNCTION s1.f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 > 0';
CREATE FUNCTION s2.f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 < 0';
CREATE POLICY pf ON t2 USING (s1.f(a));
SET search_path = public, s1;
SELECT ddl FROM pg_get_policy_ddl('t2', 'pf') AS ddl;
-- CREATE POLICY pf ON public.t2 USING (f(a)); -- s1. dropped
SET search_path = public, s2;
CREATE POLICY pf ON t2 USING (f(a)); -- now s2.f,
opposite meaning
So within one statement the ON clause is always qualified but the expression
body isn't -- and as Phil noted, the pretty flag can't fix this, since
pg_get_expr qualifies by search_path visibility regardless of pretty.
Worth settling the contract, given where this function sits. The old
pg_get_viewdef / ruledef / indexdef functions are search_path-aware and leave
qualification to the caller (pg_dump sets search_path empty around them so the
output is portable). The new pg_get_*_ddl functions committed so far (role,
database, tablespace) are on global, schemaless objects, so the question never
came up -- pg_get_policy_ddl is the first of the family whose output embeds
schema-qualifiable references. Your own pg_get_table_ddl patch already hit
this and deparses under a controlled search_path (narrowed to pg_catalog), so
the most consistent fix is to do the same here (NewGUCNestLevel + set_config);
an empty search_path already yields the fully-qualified s1.f(a). If instead
the intent is to follow the caller's search_path, that's fine too, but it
should be documented (and SET search_path = '' noted as the way to get
portable DDL).
2) The doc calls the second parameter policy_name, but the actual argument is
policyname, so the documented named-argument call fails:
SELECT * FROM pg_get_policy_ddl("table" => 't'::regclass,
policy_name => 'p_all');
ERROR: function pg_get_policy_ddl(table => regclass, policy_name
=> unknown) does not exist
Regards,
Rui
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-06 11:28 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-08 16:58 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
@ 2026-07-09 10:57 ` Akshay Joshi <[email protected]>
2026-07-12 14:38 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Akshay Joshi @ 2026-07-09 10:57 UTC (permalink / raw)
To: Rui Zhao <[email protected]>; +Cc: solai v <[email protected]>; Ilmar Y <[email protected]>; [email protected]
Thanks Rui for the review. I have tried to fix the issues you raised.
The v16 patch is ready for review/commit.
On Wed, Jul 8, 2026 at 10:28 PM Rui Zhao <[email protected]> wrote:
> Hi Akshay,
>
> I tested v15 on current master (57f93af36f): builds, make check passes, and
> basic reconstruction, pretty mode, NULL handling, and default-clause
> omission
> all look good. Two things.
>
> 1) Back in the v3/v4 discussion you decided the ON <table> name should
> always
> be schema-qualified for safety (the pg_get_triggerdef_worker thread with
> jian he), and that pretty here only controls formatting, not schema (Phil's
> point). That reasoning didn't reach the USING / WITH CHECK expressions,
> though: object references inside them are still qualified only by the
> caller's
> search_path, so they can lose their schema and rebind to a different object
> when the DDL is replayed elsewhere:
>
> CREATE FUNCTION s1.f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 > 0';
> CREATE FUNCTION s2.f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 < 0';
> CREATE POLICY pf ON t2 USING (s1.f(a));
>
> SET search_path = public, s1;
> SELECT ddl FROM pg_get_policy_ddl('t2', 'pf') AS ddl;
> -- CREATE POLICY pf ON public.t2 USING (f(a)); -- s1. dropped
>
> SET search_path = public, s2;
> CREATE POLICY pf ON t2 USING (f(a)); -- now s2.f,
> opposite meaning
>
> So within one statement the ON clause is always qualified but the
> expression
> body isn't -- and as Phil noted, the pretty flag can't fix this, since
> pg_get_expr qualifies by search_path visibility regardless of pretty.
>
> Worth settling the contract, given where this function sits. The old
> pg_get_viewdef / ruledef / indexdef functions are search_path-aware and
> leave
> qualification to the caller (pg_dump sets search_path empty around them so
> the
> output is portable). The new pg_get_*_ddl functions committed so far (role,
> database, tablespace) are on global, schemaless objects, so the question
> never
> came up -- pg_get_policy_ddl is the first of the family whose output embeds
> schema-qualifiable references. Your own pg_get_table_ddl patch already hit
> this and deparses under a controlled search_path (narrowed to pg_catalog),
> so
> the most consistent fix is to do the same here (NewGUCNestLevel +
> set_config);
> an empty search_path already yields the fully-qualified s1.f(a). If instead
> the intent is to follow the caller's search_path, that's fine too, but it
> should be documented (and SET search_path = '' noted as the way to get
> portable DDL).
>
> 2) The doc calls the second parameter policy_name, but the actual argument
> is
> policyname, so the documented named-argument call fails:
>
> SELECT * FROM pg_get_policy_ddl("table" => 't'::regclass,
> policy_name => 'p_all');
> ERROR: function pg_get_policy_ddl(table => regclass, policy_name
> => unknown) does not exist
>
> Regards,
> Rui
>
Attachments:
[application/octet-stream] v16-0001-Add-pg_get_policy_ddl-to-reconstruct-CREATE.patch (34.0K, ../../CANxoLDd4uL=EU+4cGVWp1F9oqVaye4uybUkJhhS7QfeHaqeDXQ@mail.gmail.com/3-v16-0001-Add-pg_get_policy_ddl-to-reconstruct-CREATE.patch)
download | inline diff:
From 4d82fcf9b04607804ecdcc4b3d90d57ac12ca671 Mon Sep 17 00:00:00 2001
From: Akshay Joshi <[email protected]>
Date: Fri, 22 May 2026 18:18:07 +0530
Subject: [PATCH v16] Add pg_get_policy_ddl() to reconstruct CREATE POLICY
statements
pg_get_policy_ddl(table regclass,
policyname name,
pretty bool DEFAULT false)
RETURNS SETOF text
reconstructs the CREATE POLICY statement for the named row-level
security policy on the specified table. Although a single statement is
produced, the function returns a SETOF text result so its calling
convention matches the rest of the pg_get_*_ddl family.
The reconstructed DDL includes all clauses of the CREATE POLICY syntax:
the policy's permissiveness (AS RESTRICTIVE), command type (FOR
SELECT/INSERT/UPDATE/DELETE), role list (TO <roles>), USING
qualification, and WITH CHECK expression. Clauses whose value equals
the parser's default -- PERMISSIVE, FOR ALL, and TO PUBLIC -- are
omitted from the output, matching the convention used by pg_get_indexdef,
pg_get_constraintdef, pg_get_viewdef, and similar functions. The result
is therefore semantically equivalent to the original DDL, not lexically
identical.
The pretty parameter controls output formatting and defaults to false,
following the same convention as pg_get_role_ddl, pg_get_tablespace_ddl,
and pg_get_database_ddl. NULL passed explicitly for pretty is treated
as false.
NULL inputs for the table or policy name yield no rows. An invalid
relation name surfaces the regclass resolution error; a non-existent
policy raises an explicit "policy ... does not exist" error.
Usage examples:
-- compact form (default)
SELECT * FROM pg_get_policy_ddl('rls_table', 'pol1');
SELECT * FROM pg_get_policy_ddl(16564, 'pol1');
-- pretty-printed form
SELECT * FROM pg_get_policy_ddl('rls_table', 'pol1', true);
Regression coverage is added to src/test/regress/sql/rowsecurity.sql
and exercises all valid combinations of the CREATE POLICY syntax:
PERMISSIVE/RESTRICTIVE, all FOR command variants (ALL/SELECT/INSERT/
UPDATE/DELETE), multi-role TO lists, USING-only, WITH CHECK-only, and
USING+WITH CHECK policies for both ALL and UPDATE commands (the only
two that accept both), RESTRICTIVE on a specific command type,
subquery expressions, pretty and non-pretty output, all boolean
representations for the pretty argument (true/false, on/off, 1/0),
NULL and error paths, and a round-trip test that drops and re-executes
the generated DDL.
Author: Akshay Joshi <[email protected]>
Reviewed-by: Rui Zhao <[email protected]>
---
doc/src/sgml/func/func-info.sgml | 19 ++
src/backend/utils/adt/ddlutils.c | 274 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +
src/test/regress/expected/rowsecurity.out | 315 ++++++++++++++++++++++
src/test/regress/sql/rowsecurity.sql | 150 +++++++++++
5 files changed, 764 insertions(+)
diff --git a/doc/src/sgml/func/func-info.sgml b/doc/src/sgml/func/func-info.sgml
index 69ef3857cfa..cfdef22ad5a 100644
--- a/doc/src/sgml/func/func-info.sgml
+++ b/doc/src/sgml/func/func-info.sgml
@@ -3960,6 +3960,25 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
is false, the <literal>OWNER</literal> clause is omitted.
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_policy_ddl</primary>
+ </indexterm>
+ <function>pg_get_policy_ddl</function>
+ ( <parameter>table</parameter> <type>regclass</type>,
+ <parameter>policyname</parameter> <type>name</type>
+ <optional>, <parameter>pretty</parameter> <type>boolean</type>
+ </optional> )
+ <returnvalue>setof text</returnvalue>
+ </para>
+ <para>
+ Reconstructs the <command>CREATE POLICY</command> statement for the
+ named row-level security policy on the specified table. The result
+ is returned as a single row. If <parameter>pretty</parameter> is
+ true, the output is formatted for readability; the default is false.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/src/backend/utils/adt/ddlutils.c b/src/backend/utils/adt/ddlutils.c
index a70f1c28655..2ada0e25fde 100644
--- a/src/backend/utils/adt/ddlutils.c
+++ b/src/backend/utils/adt/ddlutils.c
@@ -26,6 +26,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_database.h"
#include "catalog/pg_db_role_setting.h"
+#include "catalog/pg_policy.h"
#include "catalog/pg_tablespace.h"
#include "commands/tablespace.h"
#include "common/relpath.h"
@@ -56,6 +57,9 @@ static List *pg_get_tablespace_ddl_internal(Oid tsid, bool pretty, bool no_owner
static Datum pg_get_tablespace_ddl_srf(FunctionCallInfo fcinfo, Oid tsid);
static List *pg_get_database_ddl_internal(Oid dbid, bool pretty,
bool no_owner, bool no_tablespace);
+static List *pg_get_policy_ddl_internal(Oid tableID, const char *policyName,
+ bool pretty);
+static const char *get_policy_cmd_name(char cmd);
/*
@@ -975,3 +979,273 @@ pg_get_database_ddl(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
}
+
+/*
+ * get_policy_cmd_name
+ * Map a non-ALL pg_policy.polcmd char to its SQL keyword.
+ *
+ * Callers must guard against '*' (ALL) themselves; it is not passed here.
+ */
+static const char *
+get_policy_cmd_name(char cmd)
+{
+ switch (cmd)
+ {
+ case ACL_SELECT_CHR:
+ return "SELECT";
+ case ACL_INSERT_CHR:
+ return "INSERT";
+ case ACL_UPDATE_CHR:
+ return "UPDATE";
+ case ACL_DELETE_CHR:
+ return "DELETE";
+ default:
+ elog(ERROR, "unrecognized policy command: %d", (int) cmd);
+ return NULL; /* keep compiler quiet */
+ }
+}
+
+/*
+ * pg_get_policy_ddl_internal
+ * Generate the DDL statement to recreate a row-level security policy.
+ *
+ * Returns a List containing a single palloc'd string with the CREATE POLICY
+ * statement. Returning a List keeps the calling convention consistent with
+ * the rest of the pg_get_*_ddl family even though only one row is produced.
+ */
+static List *
+pg_get_policy_ddl_internal(Oid tableID, const char *policyName, bool pretty)
+{
+ Relation pgPolicyRel;
+ HeapTuple tuplePolicy;
+ Form_pg_policy policyForm;
+ ScanKeyData skey[2];
+ SysScanDesc sscan;
+ StringInfoData buf;
+ Datum valueDatum;
+ bool attrIsNull;
+ char *relname;
+ char *nspname;
+ char *targetTable;
+ int save_nestlevel;
+ List *statements = NIL;
+
+ /* Validate that the relation exists and build its qualified name. */
+ relname = get_rel_name(tableID);
+ if (relname == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("relation with OID %u does not exist", tableID)));
+
+ nspname = get_namespace_name(get_rel_namespace(tableID));
+ if (nspname == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_SCHEMA),
+ errmsg("schema for relation with OID %u does not exist",
+ tableID)));
+
+ targetTable = quote_qualified_identifier(nspname, relname);
+ pfree(relname);
+ pfree(nspname);
+
+ pgPolicyRel = table_open(PolicyRelationId, AccessShareLock);
+
+ /* Set key - policy's relation id. */
+ ScanKeyInit(&skey[0],
+ Anum_pg_policy_polrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tableID));
+
+ /* Set key - policy's name. */
+ ScanKeyInit(&skey[1],
+ Anum_pg_policy_polname,
+ BTEqualStrategyNumber, F_NAMEEQ,
+ CStringGetDatum(policyName));
+
+ sscan = systable_beginscan(pgPolicyRel,
+ PolicyPolrelidPolnameIndexId, true, NULL, 2,
+ skey);
+
+ tuplePolicy = systable_getnext(sscan);
+ if (!HeapTupleIsValid(tuplePolicy))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("policy \"%s\" for table \"%s\" does not exist",
+ policyName, targetTable)));
+
+ policyForm = (Form_pg_policy) GETSTRUCT(tuplePolicy);
+
+ initStringInfo(&buf);
+
+ /* Build the CREATE POLICY statement */
+ appendStringInfo(&buf, "CREATE POLICY %s ON %s",
+ quote_identifier(policyName),
+ targetTable);
+ pfree(targetTable);
+
+ /*
+ * Emit AS RESTRICTIVE only when it differs from the default (PERMISSIVE).
+ */
+ if (!policyForm->polpermissive)
+ append_ddl_option(&buf, pretty, 4, "AS RESTRICTIVE");
+
+ /*
+ * Emit FOR <cmd> only when it differs from the default (ALL, encoded as
+ * '*').
+ */
+ if (policyForm->polcmd != '*')
+ append_ddl_option(&buf, pretty, 4, "FOR %s",
+ get_policy_cmd_name(policyForm->polcmd));
+
+ /*
+ * Emit TO <roles> only when it differs from the default (PUBLIC). PUBLIC
+ * is encoded in polroles as a single InvalidOid element, so we omit the
+ * clause whenever every entry is InvalidOid. polroles is always non-NULL.
+ */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polroles,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ Assert(!attrIsNull);
+ {
+ ArrayType *policy_roles = DatumGetArrayTypeP(valueDatum);
+ int nitems = ARR_DIMS(policy_roles)[0];
+ Oid *roles = (Oid *) ARR_DATA_PTR(policy_roles);
+ StringInfoData role_names;
+
+ initStringInfo(&role_names);
+
+ for (int i = 0; i < nitems; i++)
+ {
+ if (OidIsValid(roles[i]))
+ {
+ char *rolename = GetUserNameFromId(roles[i], false);
+
+ if (role_names.len > 0)
+ appendStringInfoString(&role_names, ", ");
+ appendStringInfoString(&role_names, quote_identifier(rolename));
+ pfree(rolename);
+ }
+ }
+
+ if (role_names.len > 0)
+ append_ddl_option(&buf, pretty, 4, "TO %s", role_names.data);
+
+ pfree(role_names.data);
+ }
+
+ /*
+ * Deparse USING and WITH CHECK expressions under an empty search_path so
+ * that all object references are fully schema-qualified. This makes the
+ * reconstructed DDL portable regardless of the caller's search_path
+ * setting, consistent with how pg_dump handles pg_get_viewdef and similar
+ * functions.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ (void) set_config_option("search_path", "",
+ PGC_USERSET, PGC_S_SESSION,
+ GUC_ACTION_SAVE, true, 0, false);
+
+ /* USING expression */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polqual,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ if (!attrIsNull)
+ {
+ Datum expr;
+
+ expr = DirectFunctionCall3(pg_get_expr_ext,
+ valueDatum,
+ ObjectIdGetDatum(policyForm->polrelid),
+ BoolGetDatum(pretty));
+ append_ddl_option(&buf, pretty, 4, "USING (%s)",
+ TextDatumGetCString(expr));
+ }
+
+ /* WITH CHECK expression */
+ valueDatum = heap_getattr(tuplePolicy,
+ Anum_pg_policy_polwithcheck,
+ RelationGetDescr(pgPolicyRel),
+ &attrIsNull);
+ if (!attrIsNull)
+ {
+ Datum expr;
+
+ expr = DirectFunctionCall3(pg_get_expr_ext,
+ valueDatum,
+ ObjectIdGetDatum(policyForm->polrelid),
+ BoolGetDatum(pretty));
+ append_ddl_option(&buf, pretty, 4, "WITH CHECK (%s)",
+ TextDatumGetCString(expr));
+ }
+
+ AtEOXact_GUC(false, save_nestlevel);
+
+ appendStringInfoChar(&buf, ';');
+
+ statements = lappend(statements, pstrdup(buf.data));
+
+ systable_endscan(sscan);
+ table_close(pgPolicyRel, AccessShareLock);
+ pfree(buf.data);
+
+ return statements;
+}
+
+/*
+ * pg_get_policy_ddl
+ * Return DDL to recreate a row-level security policy as a single text row.
+ */
+Datum
+pg_get_policy_ddl(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ List *statements;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ MemoryContext oldcontext;
+ Oid tableID;
+ Name policyName;
+ bool pretty;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+ if (PG_ARGISNULL(0) || PG_ARGISNULL(1))
+ {
+ MemoryContextSwitchTo(oldcontext);
+ SRF_RETURN_DONE(funcctx);
+ }
+
+ tableID = PG_GETARG_OID(0);
+ policyName = PG_GETARG_NAME(1);
+ pretty = !PG_ARGISNULL(2) && PG_GETARG_BOOL(2);
+
+ statements = pg_get_policy_ddl_internal(tableID,
+ NameStr(*policyName),
+ pretty);
+ funcctx->user_fctx = statements;
+ funcctx->max_calls = list_length(statements);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
+ statements = (List *) funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ char *stmt;
+
+ stmt = list_nth(statements, funcctx->call_cntr);
+
+ SRF_RETURN_NEXT(funcctx, CStringGetTextDatum(stmt));
+ }
+ else
+ {
+ list_free_deep(statements);
+ SRF_RETURN_DONE(funcctx);
+ }
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..cd029b483f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8634,6 +8634,12 @@
proargtypes => 'regdatabase bool bool bool',
proargnames => '{database,pretty,owner,tablespace}',
proargdefaults => '{false,true,true}', prosrc => 'pg_get_database_ddl' },
+{ oid => '9683', descr => 'get DDL to recreate a row-level security policy',
+ proname => 'pg_get_policy_ddl', prorows => '1', proisstrict => 'f',
+ proretset => 't', provolatile => 's', pronargdefaults => '1',
+ prorettype => 'text', proargtypes => 'regclass name bool',
+ proargnames => '{table,policyname,pretty}', proargdefaults => '{false}',
+ prosrc => 'pg_get_policy_ddl' },
{ oid => '2509',
descr => 'deparse an encoded expression with pretty-print option',
proname => 'pg_get_expr', provolatile => 's', prorettype => 'text',
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 3a5e82c35bd..da174eeadbf 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -5195,6 +5195,321 @@ reset rls_test.blah;
drop function rls_f(text);
drop table rls_t, test_t;
--
+-- Test for pg_get_policy_ddl(table, policy_name[, pretty]) function.
+--
+CREATE TABLE rls_tbl_1 (
+ did int primary key,
+ cid int,
+ dlevel int not null,
+ dauthor name,
+ dtitle text
+);
+GRANT ALL ON rls_tbl_1 TO public;
+CREATE TABLE rls_tbl_2 (
+ pguser name primary key,
+ seclv int
+);
+GRANT SELECT ON rls_tbl_2 TO public;
+-- Test PERMISSIVE and RESTRICTIVE
+CREATE POLICY rls_p1 ON rls_tbl_1 AS PERMISSIVE
+ USING (dlevel <= (SELECT seclv FROM rls_tbl_2 WHERE pguser = current_user));
+CREATE POLICY rls_p2 ON rls_tbl_1 AS RESTRICTIVE USING (cid <> 44 AND cid < 50);
+-- Test FOR ALL | SELECT | INSERT | UPDATE | DELETE
+CREATE POLICY rls_p3 ON rls_tbl_1 FOR ALL USING (dauthor = current_user);
+CREATE POLICY rls_p4 ON rls_tbl_1 FOR SELECT USING (cid % 2 = 0);
+CREATE POLICY rls_p5 ON rls_tbl_1 FOR INSERT WITH CHECK (cid % 2 = 1);
+CREATE POLICY rls_p6 ON rls_tbl_1 FOR UPDATE USING (cid % 2 = 0);
+CREATE POLICY rls_p7 ON rls_tbl_1 FOR DELETE USING (cid < 8);
+-- Test TO { role_name ... }
+CREATE POLICY rls_p8 ON rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+CREATE POLICY rls_p9 ON rls_tbl_1 TO regress_rls_exempt_user WITH CHECK (cid = (SELECT seclv FROM rls_tbl_2));
+-- Test UPDATE policy with both USING and WITH CHECK
+CREATE POLICY rls_p10 ON rls_tbl_1 FOR UPDATE
+ USING (dlevel > 0) WITH CHECK (dlevel < 100);
+-- Test ALL policy with both USING and WITH CHECK (AS PERMISSIVE and FOR ALL are defaults, omitted)
+CREATE POLICY rls_p11 ON rls_tbl_1
+ USING (dlevel >= 1) WITH CHECK (dlevel <= 99);
+-- Test RESTRICTIVE on a specific command
+CREATE POLICY rls_p12 ON rls_tbl_1 AS RESTRICTIVE FOR SELECT
+ USING (cid > 0);
+-- NULL inputs should return no rows
+SELECT count(*) FROM pg_get_policy_ddl(NULL, 'rls_p1');
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM pg_get_policy_ddl('rls_tbl_1', NULL);
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM pg_get_policy_ddl(NULL, NULL);
+ count
+-------
+ 0
+(1 row)
+
+-- Table does not exist
+SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1');
+ERROR: relation "nonexistent_tbl" does not exist
+LINE 1: SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1')...
+ ^
+-- Policy does not exist
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'nonexistent_pol');
+ERROR: policy "nonexistent_pol" for table "regress_rls_schema.rls_tbl_1" does not exist
+-- Without pretty formatting (default); also verify explicit false equals default
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1');
+ pg_get_policy_ddl
+-------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p1 ON regress_rls_schema.rls_tbl_1 USING ((dlevel <= ( SELECT rls_tbl_2.seclv+
+ FROM regress_rls_schema.rls_tbl_2 +
+ WHERE (rls_tbl_2.pguser = CURRENT_USER))));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p2 ON regress_rls_schema.rls_tbl_1 AS RESTRICTIVE USING (((cid <> 44) AND (cid < 50)));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------
+ CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p4 ON regress_rls_schema.rls_tbl_1 FOR SELECT USING (((cid % 2) = 0));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p5 ON regress_rls_schema.rls_tbl_1 FOR INSERT WITH CHECK (((cid % 2) = 1));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p6 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING (((cid % 2) = 0));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7');
+ pg_get_policy_ddl
+------------------------------------------------------------------------------------
+ CREATE POLICY rls_p7 ON regress_rls_schema.rls_tbl_1 FOR DELETE USING ((cid < 8));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p8 ON regress_rls_schema.rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9');
+ pg_get_policy_ddl
+-----------------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p9 ON regress_rls_schema.rls_tbl_1 TO regress_rls_exempt_user WITH CHECK ((cid = ( SELECT rls_tbl_2.seclv+
+ FROM regress_rls_schema.rls_tbl_2)));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10');
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p11 ON regress_rls_schema.rls_tbl_1 USING ((dlevel >= 1)) WITH CHECK ((dlevel <= 99));
+(1 row)
+
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12');
+ pg_get_policy_ddl
+----------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p12 ON regress_rls_schema.rls_tbl_1 AS RESTRICTIVE FOR SELECT USING ((cid > 0));
+(1 row)
+
+-- Explicit false must match omitting the argument
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', false);
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+-- NULL pretty must also default to non-pretty
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', NULL);
+ pg_get_policy_ddl
+--------------------------------------------------------------------------------------------------------------------
+ CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1 FOR UPDATE USING ((dlevel > 0)) WITH CHECK ((dlevel < 100));
+(1 row)
+
+-- pretty accepts all standard boolean representations: true/false, on/off, 1/0
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'on'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '1'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'off'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '0'::bool);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1 USING ((dauthor = CURRENT_USER));
+(1 row)
+\pset format aligned
+-- With pretty formatting
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p1 ON regress_rls_schema.rls_tbl_1
+ USING (dlevel <= (( SELECT rls_tbl_2.seclv
+ FROM regress_rls_schema.rls_tbl_2
+ WHERE rls_tbl_2.pguser = CURRENT_USER)));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p2 ON regress_rls_schema.rls_tbl_1
+ AS RESTRICTIVE
+ USING (cid <> 44 AND cid < 50);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p3 ON regress_rls_schema.rls_tbl_1
+ USING (dauthor = CURRENT_USER);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p4 ON regress_rls_schema.rls_tbl_1
+ FOR SELECT
+ USING ((cid % 2) = 0);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p5 ON regress_rls_schema.rls_tbl_1
+ FOR INSERT
+ WITH CHECK ((cid % 2) = 1);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p6 ON regress_rls_schema.rls_tbl_1
+ FOR UPDATE
+ USING ((cid % 2) = 0);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p7 ON regress_rls_schema.rls_tbl_1
+ FOR DELETE
+ USING (cid < 8);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p8 ON regress_rls_schema.rls_tbl_1
+ TO regress_rls_dave, regress_rls_alice
+ USING (true);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p9 ON regress_rls_schema.rls_tbl_1
+ TO regress_rls_exempt_user
+ WITH CHECK (cid = (( SELECT rls_tbl_2.seclv
+ FROM regress_rls_schema.rls_tbl_2)));
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p10 ON regress_rls_schema.rls_tbl_1
+ FOR UPDATE
+ USING (dlevel > 0)
+ WITH CHECK (dlevel < 100);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p11 ON regress_rls_schema.rls_tbl_1
+ USING (dlevel >= 1)
+ WITH CHECK (dlevel <= 99);
+(1 row)
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12', true);
+pg_get_policy_ddl
+CREATE POLICY rls_p12 ON regress_rls_schema.rls_tbl_1
+ AS RESTRICTIVE
+ FOR SELECT
+ USING (cid > 0);
+(1 row)
+\pset format aligned
+-- Round-trip: the generated DDL must be re-executable, including for atomic
+-- boolean expressions that pg_get_expr() does not parenthesize.
+CREATE TABLE rls_rt (a int);
+CREATE POLICY rt_true ON rls_rt USING (true);
+CREATE POLICY rt_false ON rls_rt FOR INSERT WITH CHECK (false);
+CREATE TEMP TABLE rt_ddl AS
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_true') AS ddl
+ UNION ALL
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_false');
+DROP POLICY rt_true ON rls_rt;
+DROP POLICY rt_false ON rls_rt;
+SELECT ddl FROM rt_ddl ORDER BY ddl \gexec
+CREATE POLICY rt_false ON regress_rls_schema.rls_rt FOR INSERT WITH CHECK (false);
+CREATE POLICY rt_true ON regress_rls_schema.rls_rt USING (true);
+SELECT polname FROM pg_policy WHERE polrelid = 'rls_rt'::regclass ORDER BY polname;
+ polname
+----------
+ rt_false
+ rt_true
+(2 rows)
+
+DROP TABLE rls_rt;
+-- Schema-qualification: function references in expressions must be
+-- fully qualified regardless of the caller's search_path.
+CREATE SCHEMA rls_s1;
+CREATE SCHEMA rls_s2;
+CREATE FUNCTION rls_s1.rls_f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 > 0';
+CREATE FUNCTION rls_s2.rls_f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 < 0';
+CREATE TABLE rls_tbl_3 (a int);
+CREATE POLICY rls_pf ON rls_tbl_3 USING (rls_s1.rls_f(a));
+-- With rls_s1 in path, rls_f should still appear schema-qualified in DDL.
+SET search_path = regress_rls_schema, rls_s1;
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_3', 'rls_pf');
+pg_get_policy_ddl
+CREATE POLICY rls_pf ON regress_rls_schema.rls_tbl_3 USING (rls_s1.rls_f(a));
+(1 row)
+\pset format aligned
+-- Restore the test's search_path before cleanup.
+SET search_path = regress_rls_schema;
+DROP POLICY rls_pf ON rls_tbl_3;
+DROP TABLE rls_tbl_3;
+DROP FUNCTION rls_s1.rls_f(int);
+DROP FUNCTION rls_s2.rls_f(int);
+DROP SCHEMA rls_s1;
+DROP SCHEMA rls_s2;
+-- Clean up objects created for testing pg_get_policy_ddl function.
+DROP POLICY rls_p1 ON rls_tbl_1;
+DROP POLICY rls_p2 ON rls_tbl_1;
+DROP POLICY rls_p3 ON rls_tbl_1;
+DROP POLICY rls_p4 ON rls_tbl_1;
+DROP POLICY rls_p5 ON rls_tbl_1;
+DROP POLICY rls_p6 ON rls_tbl_1;
+DROP POLICY rls_p7 ON rls_tbl_1;
+DROP POLICY rls_p8 ON rls_tbl_1;
+DROP POLICY rls_p9 ON rls_tbl_1;
+DROP POLICY rls_p10 ON rls_tbl_1;
+DROP POLICY rls_p11 ON rls_tbl_1;
+DROP POLICY rls_p12 ON rls_tbl_1;
+DROP TABLE rls_tbl_1;
+DROP TABLE rls_tbl_2;
+--
-- Clean up objects
--
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/rowsecurity.sql b/src/test/regress/sql/rowsecurity.sql
index 6b3566271df..cdc50d74cde 100644
--- a/src/test/regress/sql/rowsecurity.sql
+++ b/src/test/regress/sql/rowsecurity.sql
@@ -2598,6 +2598,156 @@ reset rls_test.blah;
drop function rls_f(text);
drop table rls_t, test_t;
+--
+-- Test for pg_get_policy_ddl(table, policy_name[, pretty]) function.
+--
+CREATE TABLE rls_tbl_1 (
+ did int primary key,
+ cid int,
+ dlevel int not null,
+ dauthor name,
+ dtitle text
+);
+GRANT ALL ON rls_tbl_1 TO public;
+CREATE TABLE rls_tbl_2 (
+ pguser name primary key,
+ seclv int
+);
+GRANT SELECT ON rls_tbl_2 TO public;
+
+-- Test PERMISSIVE and RESTRICTIVE
+CREATE POLICY rls_p1 ON rls_tbl_1 AS PERMISSIVE
+ USING (dlevel <= (SELECT seclv FROM rls_tbl_2 WHERE pguser = current_user));
+CREATE POLICY rls_p2 ON rls_tbl_1 AS RESTRICTIVE USING (cid <> 44 AND cid < 50);
+
+-- Test FOR ALL | SELECT | INSERT | UPDATE | DELETE
+CREATE POLICY rls_p3 ON rls_tbl_1 FOR ALL USING (dauthor = current_user);
+CREATE POLICY rls_p4 ON rls_tbl_1 FOR SELECT USING (cid % 2 = 0);
+CREATE POLICY rls_p5 ON rls_tbl_1 FOR INSERT WITH CHECK (cid % 2 = 1);
+CREATE POLICY rls_p6 ON rls_tbl_1 FOR UPDATE USING (cid % 2 = 0);
+CREATE POLICY rls_p7 ON rls_tbl_1 FOR DELETE USING (cid < 8);
+
+-- Test TO { role_name ... }
+CREATE POLICY rls_p8 ON rls_tbl_1 TO regress_rls_dave, regress_rls_alice USING (true);
+CREATE POLICY rls_p9 ON rls_tbl_1 TO regress_rls_exempt_user WITH CHECK (cid = (SELECT seclv FROM rls_tbl_2));
+
+-- Test UPDATE policy with both USING and WITH CHECK
+CREATE POLICY rls_p10 ON rls_tbl_1 FOR UPDATE
+ USING (dlevel > 0) WITH CHECK (dlevel < 100);
+
+-- Test ALL policy with both USING and WITH CHECK (AS PERMISSIVE and FOR ALL are defaults, omitted)
+CREATE POLICY rls_p11 ON rls_tbl_1
+ USING (dlevel >= 1) WITH CHECK (dlevel <= 99);
+
+-- Test RESTRICTIVE on a specific command
+CREATE POLICY rls_p12 ON rls_tbl_1 AS RESTRICTIVE FOR SELECT
+ USING (cid > 0);
+
+-- NULL inputs should return no rows
+SELECT count(*) FROM pg_get_policy_ddl(NULL, 'rls_p1');
+SELECT count(*) FROM pg_get_policy_ddl('rls_tbl_1', NULL);
+SELECT count(*) FROM pg_get_policy_ddl(NULL, NULL);
+
+-- Table does not exist
+SELECT * FROM pg_get_policy_ddl('nonexistent_tbl', 'rls_p1');
+-- Policy does not exist
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'nonexistent_pol');
+
+-- Without pretty formatting (default); also verify explicit false equals default
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11');
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12');
+-- Explicit false must match omitting the argument
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', false);
+-- NULL pretty must also default to non-pretty
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', NULL);
+
+-- pretty accepts all standard boolean representations: true/false, on/off, 1/0
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'on'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '1'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', 'off'::bool);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', '0'::bool);
+\pset format aligned
+
+-- With pretty formatting
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p1', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p2', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p3', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p4', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p5', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p6', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p7', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p8', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p9', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p10', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p11', true);
+SELECT * FROM pg_get_policy_ddl('rls_tbl_1', 'rls_p12', true);
+\pset format aligned
+
+-- Round-trip: the generated DDL must be re-executable, including for atomic
+-- boolean expressions that pg_get_expr() does not parenthesize.
+CREATE TABLE rls_rt (a int);
+CREATE POLICY rt_true ON rls_rt USING (true);
+CREATE POLICY rt_false ON rls_rt FOR INSERT WITH CHECK (false);
+CREATE TEMP TABLE rt_ddl AS
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_true') AS ddl
+ UNION ALL
+ SELECT pg_get_policy_ddl('rls_rt', 'rt_false');
+DROP POLICY rt_true ON rls_rt;
+DROP POLICY rt_false ON rls_rt;
+SELECT ddl FROM rt_ddl ORDER BY ddl \gexec
+SELECT polname FROM pg_policy WHERE polrelid = 'rls_rt'::regclass ORDER BY polname;
+DROP TABLE rls_rt;
+
+-- Schema-qualification: function references in expressions must be
+-- fully qualified regardless of the caller's search_path.
+CREATE SCHEMA rls_s1;
+CREATE SCHEMA rls_s2;
+CREATE FUNCTION rls_s1.rls_f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 > 0';
+CREATE FUNCTION rls_s2.rls_f(int) RETURNS bool LANGUAGE sql AS 'SELECT $1 < 0';
+CREATE TABLE rls_tbl_3 (a int);
+CREATE POLICY rls_pf ON rls_tbl_3 USING (rls_s1.rls_f(a));
+-- With rls_s1 in path, rls_f should still appear schema-qualified in DDL.
+SET search_path = regress_rls_schema, rls_s1;
+\pset format unaligned
+SELECT * FROM pg_get_policy_ddl('rls_tbl_3', 'rls_pf');
+\pset format aligned
+-- Restore the test's search_path before cleanup.
+SET search_path = regress_rls_schema;
+DROP POLICY rls_pf ON rls_tbl_3;
+DROP TABLE rls_tbl_3;
+DROP FUNCTION rls_s1.rls_f(int);
+DROP FUNCTION rls_s2.rls_f(int);
+DROP SCHEMA rls_s1;
+DROP SCHEMA rls_s2;
+
+-- Clean up objects created for testing pg_get_policy_ddl function.
+DROP POLICY rls_p1 ON rls_tbl_1;
+DROP POLICY rls_p2 ON rls_tbl_1;
+DROP POLICY rls_p3 ON rls_tbl_1;
+DROP POLICY rls_p4 ON rls_tbl_1;
+DROP POLICY rls_p5 ON rls_tbl_1;
+DROP POLICY rls_p6 ON rls_tbl_1;
+DROP POLICY rls_p7 ON rls_tbl_1;
+DROP POLICY rls_p8 ON rls_tbl_1;
+DROP POLICY rls_p9 ON rls_tbl_1;
+DROP POLICY rls_p10 ON rls_tbl_1;
+DROP POLICY rls_p11 ON rls_tbl_1;
+DROP POLICY rls_p12 ON rls_tbl_1;
+DROP TABLE rls_tbl_1;
+DROP TABLE rls_tbl_2;
+
--
-- Clean up objects
--
--
2.51.0
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-06 11:28 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-08 16:58 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
2026-07-09 10:57 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
@ 2026-07-12 14:38 ` Rui Zhao <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Rui Zhao @ 2026-07-12 14:38 UTC (permalink / raw)
To: Akshay Joshi <[email protected]>; +Cc: solai v <[email protected]>; Ilmar Y <[email protected]>; [email protected]
Hi Akshay,
v16 fixes both points from my last mail. Verified on master 5f14f82280:
builds, rowsecurity passes, and the s1.f/s2.f case now round-trips under a
hostile search_path. Nice that the rls_s1/rls_s2 test pins it.
I also diffed the generated DDL against pg_dump for the policies in the RLS
regression tests and pg_dump's own dump-test corpus (002_pg_dump.pl): they
match in every case except multi-role ordering (e.g. rls_p8 -- the function
keeps the declared order, TO regress_rls_dave, regress_rls_alice, while pg_dump
sorts by role OID), which doesn't matter since role order in a policy isn't
significant.
One thing left over from point 1: func-info.sgml doesn't state that object
references in the output are always schema-qualified. That's the function's
contract now, and it differs from pg_get_viewdef / ruledef (which follow the
caller's search_path), so a sentence would help. Empty search_path is also
exactly what pg_dump uses (ALWAYS_SECURE_SEARCH_PATH_SQL), so this is the
right call.
A tiny style nit, in pg_get_policy_ddl_internal():
Assert(!attrIsNull);
{
ArrayType *policy_roles = DatumGetArrayTypeP(valueDatum);
a blank line after the Assert would read a bit better.
Neither is a blocker -- looks ready to me.
Regards,
Rui
^ permalink raw reply [nested|flat] 25+ messages in thread
end of thread, other threads:[~2026-07-12 14:38 UTC | newest]
Thread overview: 25+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-11-17 10:46 RE: pgbench - doCustom cleanup Fabien COELHO <[email protected]>
2018-11-19 23:02 ` Alvaro Herrera <[email protected]>
2018-11-20 13:41 ` Fabien COELHO <[email protected]>
2018-11-20 14:27 ` Alvaro Herrera <[email protected]>
2018-11-20 15:21 ` Fabien COELHO <[email protected]>
2018-11-20 22:43 ` Alvaro Herrera <[email protected]>
2018-11-20 22:48 ` Alvaro Herrera <[email protected]>
2018-11-21 19:37 ` Alvaro Herrera <[email protected]>
2018-11-24 08:58 ` Fabien COELHO <[email protected]>
2018-11-21 18:08 ` Alvaro Herrera <[email protected]>
2019-05-28 18:15 [PATCH v17 1/7] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH v18 1/6] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH 1/6] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH v14 1/7] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH v16 1/7] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH v15 1/7] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2019-05-28 18:15 [PATCH v16 1/7] Remove pg_collation.collversion. Thomas Munro <[email protected]>
2025-04-04 08:52 Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning Amit Langote <[email protected]>
2026-07-01 09:15 Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-01 10:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-01 11:31 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement solai v <[email protected]>
2026-07-06 11:28 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-08 16:58 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
2026-07-09 10:57 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Akshay Joshi <[email protected]>
2026-07-12 14:38 ` Re: [PATCH] Add pg_get_policy_ddl() function to reconstruct CREATE POLICY statement Rui Zhao <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox