public inbox for [email protected]  

help / color / mirror / Atom feed

From: Andres Freund <[email protected]>
To: Muhammad Malik <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Thomas Munro <[email protected]>
Cc: Melanie Plageman <[email protected]>
Cc: Yura Sokolov <[email protected]>
Cc: Robert Haas <[email protected]>
Subject: Re: refactoring relation extension and BufferAlloc(), faster COPY
Date: Wed, 3 May 2023 14:25:45 -0700
Message-ID: <[email protected]> (raw)
In-Reply-To: <SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9@SJ0PR17MB5841.namprd17.prod.outlook.com>
References: <[email protected]>
	<SJ0PR17MB5841FC5FE92FBF6FA76EA07BA66C9@SJ0PR17MB5841.namprd17.prod.outlook.com>
	<[email protected]>
	<SJ0PR17MB58412FDEE3DCAC0C23C062ABA66C9@SJ0PR17MB5841.namprd17.prod.outlook.com>

Hi,

On 2023-05-03 19:29:46 +0000, Muhammad Malik wrote:
> > I use a script like:
>
> > c=16;psql -c 'DROP TABLE IF EXISTS copytest_0; CREATE TABLE copytest_0(data text not null);' && time /srv/dev/build/m-opt/src/bin/pgbench/pgbench -n -P1 -c$c -j$c -t$((1024/$c)) -f ~/tmp/copy.sql && psql -c 'TRUNCATE copytest_0'
>
> > >[1] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1, 100000)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT test);
> > >[2] COPY (SELECT repeat(random()::text, 5) FROM generate_series(1, 6*100000)) TO '/tmp/copytest_data_text.copy' WITH (FORMAT text);
>
> When I ran this script it did not insert anything into the copytest_0 table. It only generated a single copytest_data_text.copy file of size 9.236MB.
> Please help me understand how is this 'pgbench running COPY into a single table'.

That's the data generation for the file to be COPYed in. The script passed to
pgbench is just something like

COPY copytest_0 FROM '/tmp/copytest_data_text.copy';
or
COPY copytest_0 FROM '/tmp/copytest_data_binary.copy';


> Also what are the 'seconds' and 'tbl-MBs' metrics that were reported.

The total time for inserting N (1024 for the small files, 64 for the larger
ones). "tbl-MBs" is size of the resulting table, divided by time. I.e. a
measure of throughput.

Greetings,

Andres Freund

view thread (5+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: refactoring relation extension and BufferAlloc(), faster COPY
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox