pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedmultiple sessions to create index in parallel support
6+ messages / 2 participants
[nested] [flat]
* multiple sessions to create index in parallel support
@ 2022-06-21 10:18 James Pang (chaolpan) <chaolpan@cisco.com>
0 siblings, 1 reply; 6+ messages in thread
From: James Pang (chaolpan) @ 2022-06-21 10:18 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
Hi,
We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes , with only one connection , the create index automatically run in parallel (table size 40G) , then we started two connections and each session creating index on large table, but at this time, only 1 session parallel creating index, the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 session allowed to parallel index creating at the same time ?
Thanks,
James
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: multiple sessions to create index in parallel support
@ 2022-06-21 14:30 Tom Lane <tgl@sss.pgh.pa.us>
parent: James Pang (chaolpan) <chaolpan@cisco.com>
0 siblings, 1 reply; 6+ messages in thread
From: Tom Lane @ 2022-06-21 14:30 UTC (permalink / raw)
To: James Pang (chaolpan) <chaolpan@cisco.com>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes , with only one connection , the create index automatically run in parallel (table size 40G) , then we started two connections and each session creating index on large table, but at this time, only 1 session parallel creating index, the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 session allowed to parallel index creating at the same time ?
Check your settings for max_parallel_workers and
max_parallel_maintenance_workers.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* RE: multiple sessions to create index in parallel support
@ 2022-06-21 14:44 James Pang (chaolpan) <chaolpan@cisco.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 6+ messages in thread
From: James Pang (chaolpan) @ 2022-06-21 14:44 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large table show parallel 8 workers, the other session creating one index in another separate table always in serial. Even increase max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other is done in serial.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 10:30 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes , with only one connection , the create index automatically run in parallel (table size 40G) , then we started two connections and each session creating index on large table, but at this time, only 1 session parallel creating index, the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 session allowed to parallel index creating at the same time ?
Check your settings for max_parallel_workers and max_parallel_maintenance_workers.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: multiple sessions to create index in parallel support
@ 2022-06-21 15:03 Tom Lane <tgl@sss.pgh.pa.us>
parent: James Pang (chaolpan) <chaolpan@cisco.com>
0 siblings, 1 reply; 6+ messages in thread
From: Tom Lane @ 2022-06-21 15:03 UTC (permalink / raw)
To: James Pang (chaolpan) <chaolpan@cisco.com>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large table show parallel 8 workers, the other session creating one index in another separate table always in serial. Even increase max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other is done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* RE: multiple sessions to create index in parallel support
@ 2022-06-22 01:33 James Pang (chaolpan) <chaolpan@cisco.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 6+ messages in thread
From: James Pang (chaolpan) @ 2022-06-22 01:33 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally separate large tables(40gb) , either one in 8 parallel workers, but not both.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 11:04 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large table show parallel 8 workers, the other session creating one index in another separate table always in serial. Even increase max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other is done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* RE: multiple sessions to create index in parallel support
@ 2022-06-22 14:15 James Pang (chaolpan) <chaolpan@cisco.com>
parent: James Pang (chaolpan) <chaolpan@cisco.com>
0 siblings, 0 replies; 6+ messages in thread
From: James Pang (chaolpan) @ 2022-06-22 14:15 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
Sorry for confusion. This time, I add "max_parallel_workers"=32 , "max_parallel_maintenance_workers"=32 , "max_workers_process"=64 to postgresql.conf, and restart database. This time, both session runs in parallel now. Last several tests only got these parameter changes by set max_parallell_workers... in session level , for each session , only 1 session parallel creating index.
James
-----Original Message-----
From: James Pang (chaolpan)
Sent: Wednesday, June 22, 2022 9:34 AM
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin@lists.postgresql.org
Subject: RE: multiple sessions to create index in parallel support
All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally separate large tables(40gb) , either one in 8 parallel workers, but not both.
James
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Tuesday, June 21, 2022 11:04 PM
To: James Pang (chaolpan) <chaolpan@cisco.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: multiple sessions to create index in parallel support
"James Pang (chaolpan)" <chaolpan@cisco.com> writes:
> Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large table show parallel 8 workers, the other session creating one index in another separate table always in serial. Even increase max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other is done in serial.
Hmm, what about max_worker_processes ? That's an independent limit.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-06-22 14:15 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-06-21 10:18 multiple sessions to create index in parallel support James Pang (chaolpan) <chaolpan@cisco.com>
2022-06-21 14:30 ` Re: multiple sessions to create index in parallel support Tom Lane <tgl@sss.pgh.pa.us>
2022-06-21 14:44 ` RE: multiple sessions to create index in parallel support James Pang (chaolpan) <chaolpan@cisco.com>
2022-06-21 15:03 ` Re: multiple sessions to create index in parallel support Tom Lane <tgl@sss.pgh.pa.us>
2022-06-22 01:33 ` RE: multiple sessions to create index in parallel support James Pang (chaolpan) <chaolpan@cisco.com>
2022-06-22 14:15 ` RE: multiple sessions to create index in parallel support James Pang (chaolpan) <chaolpan@cisco.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox