pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: pgbench: Use COPY for client-side data generation
Date: Mon, 24 Jul 2023 04:50:01 +0000
Message-ID: <E1qNnWC-00131s-NL@gemulon.postgresql.org> (raw)

pgbench: Use COPY for client-side data generation

This commit switches the client-side data generation from INSERT queries
to COPY for the two tables pgbench_branches and pgbench_tellers.
pgbench_accounts was already using COPY.

COPY is a better interface for bulk loading or high latency connections
(this point can be countered with the option for server-side data
generation, still client-side is the default), and measurements have
proved that using it for these two other tables can lead to improvements
during initialization.  I did not notice slowdowns at large scale
numbers on a local setup, either, most of the work happening for the
accounts table.

Previously COPY was only used for the pgbench_accounts table because the
amount of data was much larger than the two other tables.  The code is
refactored so as all three tables use the same code path to execute the
COPY queries, with a callback to build data rows.

Author: Tristan Partin
Discussion: https://postgr.es/m/CSTU5P82ONZ1.19XFUGHMXHBRY@c3po

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e35cc3b3f2d081e2de3a0e077715d12b3580cc74

Modified Files
--------------
doc/src/sgml/ref/pgbench.sgml |   9 +--
src/bin/pgbench/pgbench.c     | 155 +++++++++++++++++++++++++-----------------
2 files changed, 98 insertions(+), 66 deletions(-)



Message-ID: <E1qNnWC-00131s-NL@gemulon.postgresql.org>
Permalink:  ../E1qNnWC-00131s-NL@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1qNnWC-00131s-NL@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-committers@postgresql.org
  Cc: michael@paquier.xyz, pgsql-committers@lists.postgresql.org
  Subject: Re: pgsql: pgbench: Use COPY for client-side data generation
  In-Reply-To: <E1qNnWC-00131s-NL@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox