public inbox for [email protected]
help / color / mirror / Atom feedFrom: Yugo NAGATA <[email protected]>
To: Alvaro Herrera <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Fabien COELHO <[email protected]>
Cc: [email protected]
Subject: Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error
Date: Fri, 30 Sep 2022 10:07:43 +0900
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
Hi,
On Mon, 12 Sep 2022 17:03:43 +0200
Alvaro Herrera <[email protected]> wrote:
> On 2022-Mar-28, Yugo NAGATA wrote:
>
> > On Fri, 25 Mar 2022 16:19:54 -0400
> > Tom Lane <[email protected]> wrote:
>
> > > I am not convinced this is a great idea. The current behavior is that
> > > a statement is not prepared until it's about to be executed, and I think
> > > we chose that deliberately to avoid semantic differences between prepared
> > > and not-prepared mode. For example, if a script looks like
> > >
> > > CREATE FUNCTION foo(...) ...;
> > > SELECT foo(...);
> > > DROP FUNCTION foo;
> > >
> > > trying to prepare the SELECT in advance would lead to failure.
> > >
> > > We could perhaps get away with preparing the commands within a pipeline
> > > just before we start to execute the pipeline, but it looks to me like
> > > this patch tries to prepare the entire script in advance.
>
> Maybe it would work to have one extra boolean in struct Command, indicating
> that the i-th command in the script is inside a pipeline; in -M
> prepared, issue PREPARE for each command marked with that flag ahead of
> time, and for all other commands, do as today. That way, we don't
> change behavior for anything except those commands that need the change.
Well, I still don't understand why we need to prepare only "the
commands within a pipeline" before starting pipeline. In the current
behavior, the entire script is prepared in advance just before executing
the first SQL command in the script, instead of preparing each command
one by one. This patch also prepare the entire script in advance, so
there is no behavioural change in this sense.
However, there are a few behavioural changes. One is that the preparation
is not counted in the command performance statistics as Fabien mentioned.
Another is that all meta-commands including \shell and \sleep etc. are
executed before the preparation.
To reduce impact of these changes, I updated the patch to prepare the
commands just before executing the first SQL command or \startpipeline
meta-command instead of at the beginning of the script.
Regards,
Yugo Nagata
>
> --
> Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
> "Digital and video cameras have this adjustment and film cameras don't for the
> same reason dogs and cats lick themselves: because they can." (Ken Rockwell)
--
Yugo NAGATA <[email protected]>
Attachments:
[text/x-diff] v6-pgbench-prepare.patch (3.7K, ../[email protected]/2-v6-pgbench-prepare.patch)
download | inline diff:
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index aa1a3541fe..12a268fe84 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -3070,6 +3070,30 @@ chooseScript(TState *thread)
return i - 1;
}
+/* Prepare SQL commands in the chosen script */
+static void
+prepareCommands(CState *st)
+{
+ int j;
+ Command **commands = sql_script[st->use_file].commands;
+
+ for (j = 0; commands[j] != NULL; j++)
+ {
+ PGresult *res;
+ char name[MAX_PREPARE_NAME];
+
+ if (commands[j]->type != SQL_COMMAND)
+ continue;
+ preparedStatementName(name, st->use_file, j);
+ res = PQprepare(st->con, name,
+ commands[j]->argv[0], commands[j]->argc - 1, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ pg_log_error("%s", PQerrorMessage(st->con));
+ PQclear(res);
+ }
+ st->prepared[st->use_file] = true;
+}
+
/* Send a SQL command, using the chosen querymode */
static bool
sendCommand(CState *st, Command *command)
@@ -3104,39 +3128,7 @@ sendCommand(CState *st, Command *command)
const char *params[MAX_ARGS];
if (!st->prepared[st->use_file])
- {
- int j;
- Command **commands = sql_script[st->use_file].commands;
-
- for (j = 0; commands[j] != NULL; j++)
- {
- PGresult *res;
-
- if (commands[j]->type != SQL_COMMAND)
- continue;
- preparedStatementName(name, st->use_file, j);
- if (PQpipelineStatus(st->con) == PQ_PIPELINE_OFF)
- {
- res = PQprepare(st->con, name,
- commands[j]->argv[0], commands[j]->argc - 1, NULL);
- if (PQresultStatus(res) != PGRES_COMMAND_OK)
- pg_log_error("%s", PQerrorMessage(st->con));
- PQclear(res);
- }
- else
- {
- /*
- * In pipeline mode, we use asynchronous functions. If a
- * server-side error occurs, it will be processed later
- * among the other results.
- */
- if (!PQsendPrepare(st->con, name,
- commands[j]->argv[0], commands[j]->argc - 1, NULL))
- pg_log_error("%s", PQerrorMessage(st->con));
- }
- }
- st->prepared[st->use_file] = true;
- }
+ prepareCommands(st);
getQueryParams(&st->variables, command, params);
preparedStatementName(name, st->use_file, st->command);
@@ -4376,6 +4368,19 @@ executeMetaCommand(CState *st, pg_time_usec_t *now)
commandFailed(st, "startpipeline", "cannot use pipeline mode with the simple query protocol");
return CSTATE_ABORTED;
}
+ /*
+ * Prepare SQL commands if needed.
+ *
+ * We should send Parse messages before executing /startpipeline.
+ * If a Parse message is sent in pipeline mode, a transaction starts
+ * before BEGIN is sent, and it could be a problem. For example, if
+ * "BEGIN ISOLATION LEVEL SERIALIZABLE" is sent after a transaction
+ * starts, the error
+ * "ERROR: SET TRANSACTION ISOLATION LEVEL must be called before any query"
+ * occurs.
+ */
+ else if (querymode == QUERY_PREPARED && !st->prepared[st->use_file])
+ prepareCommands(st);
if (PQpipelineStatus(st->con) != PQ_PIPELINE_OFF)
{
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 2c0dc36965..e70a7d966c 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -839,6 +839,23 @@ select 1 \gset f
}
});
+# Working \startpipeline in prepared query mode with serializable
+$node->pgbench(
+ '-t 1 -n -M prepared',
+ 0,
+ [ qr{type: .*/001_pgbench_pipeline_serializable}, qr{actually processed: 1/1} ],
+ [],
+ 'working \startpipeline with serializable',
+ {
+ '001_pgbench_pipeline_serializable' => q{
+-- test startpipeline with serializable
+\startpipeline
+BEGIN ISOLATION LEVEL SERIALIZABLE;
+} . "select 1;\n" x 10 . q{
+END;
+\endpipeline
+}
+ });
# trigger many expression errors
my @errors = (
view thread (14+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox