public inbox for [email protected]  

help / color / mirror / Atom feed
From: lakshmi <[email protected]>
To: Hayato Kuroda (Fujitsu) <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: parallel data loading for pgbench -i
Date: Mon, 13 Apr 2026 17:21:06 +0530
Message-ID: <CAEvyyThGm4NHDnfCGeCCOZ1_nrB=Eqct6y55GGuW0_UpTAsu3g@mail.gmail.com> (raw)
In-Reply-To: <OS9PR01MB12149E0E2CAAB378D264A35B1F5242@OS9PR01MB12149.jpnprd01.prod.outlook.com>
References: <[email protected]>
	<CAEvyyTj0rEsgcQOQgkARbRPbupHR_mc=TUzHBBLNzd8JByUUTw@mail.gmail.com>
	<[email protected]>
	<OS9PR01MB12149A6CEE200E1A5A88D9F14F563A@OS9PR01MB12149.jpnprd01.prod.outlook.com>
	<CAEvyyTiQqd=rv3XUxc0YEaW-feopksBveZKKjVZNeSVG=GrY+A@mail.gmail.com>
	<TYRPR01MB121560B291DA3CD262CC7A09AF568A@TYRPR01MB12156.jpnprd01.prod.outlook.com>
	<CAEvyyTjPWfvJLn3c_G_zLRffZ3=YqzMYj6c5znaNxpHyZAg3XQ@mail.gmail.com>
	<[email protected]>
	<CAEvyyTircZ-tHgap=J6Aog0CBgXp4Dqx6dHYyK1iqgfoT+8D_A@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<OS9PR01MB12149E0E2CAAB378D264A35B1F5242@OS9PR01MB12149.jpnprd01.prod.outlook.com>

Hi Hayato,

Thanks for your feedback.

I tried a few runs with different partition counts. From what I saw,
performance doesn’t always improve with more partitions—in fact, higher
partition counts increase VACUUM time and slow things down.

I also agree that having control over the number of workers (like using -j)
would help balance this better.

Regarding TRUNCATE, I noticed it’s already done earlier, so it might be
worth checking if the extra TRUNCATE is needed.

I didn’t see memory issues in my tests, but I understand it could become a
concern with many partitions.

Thanks again for the suggestions.

Best regards,
Lakshmi

On Mon, Apr 13, 2026 at 12:53 PM Hayato Kuroda (Fujitsu) <
[email protected]> wrote:

> Dear Mircea,
>
> Thanks for updating the patch. Now each worker looks like not to create
> each
> child tables, just run TRUNCATE and COPY. But I'm unclear why the TRUNCATE
> is
> needed here. Isn't they truncated in
> initGenerateDataClientSide()->initTruncateTables()
> before launching threads?
> Also, the current API is questionable. E.g., we cannot work in series if
> --partition is
> specified. And I'm afraid OOM failure may be more likely to happen if the
> table has
> many partitions.
> Is it possible that we can have -p again for the initialization? We can
> require
> partitions >= nthreads or partitions % nthreads == 0 at that time.
>
>
> Best regards,
> Hayato Kuroda
> FUJITSU LIMITED
>
>


view thread (11+ messages)  latest in thread

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: parallel data loading for pgbench -i
  In-Reply-To: <CAEvyyThGm4NHDnfCGeCCOZ1_nrB=Eqct6y55GGuW0_UpTAsu3g@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox