From: Michael Paquier <michael@paquier.xyz>
To: David G. Johnston <david.g.johnston@gmail.com>
Cc: Praneel Devisetty <devisettypraneel@gmail.com>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: REINDEXdb performance degrading gradually PG13.4
Date: Wed, 1 Jun 2022 13:37:37 +0900
Message-ID: <YpbtEfXC7X1llLk+@paquier.xyz> (raw)
In-Reply-To: <CAKFQuwZvaWqDRjc-p3bTbvT=0+UhGtYUMQJjCei2w7jJ9h0g2g@mail.gmail.com>
References: <CAHnVB4C2pawGCfNhO6eX5KL8d9BFjw5igWJ0_ogtknsxOLyfYg@mail.gmail.com>
<CAHnVB4ADj=dzzYYsZMu+pTHgTA0=2NCuKXoXbN3ceJ1dUfetHg@mail.gmail.com>
<CAKFQuwZvaWqDRjc-p3bTbvT=0+UhGtYUMQJjCei2w7jJ9h0g2g@mail.gmail.com>
On Tue, May 31, 2022 at 08:42:06AM -0700, David G. Johnston wrote:
> Building indexes is so IO heavy that the non-IO bottlenecks that exists
> likely have minimal impact on the overall times this rebuild everything
> will take. That said, I’ve never done anything at this scale before. I
> wouldn’t be too surprised if per-session cache effects are coming into play
> given the number of objects involved and the assumption that each session
> used for parallelism is persistent. I’m not sure how the parallelism works
> for managing the work queue though as it isn’t documented and I haven’t
> inspected the source code.
get_parallel_object_list() in reindexdb.c would give the idea, where
the list of tables to rebuild are ordered based on an "ORDER BY
c.relpages DESC", then the table queue is processed with its own
command, moving on to the next item once we are done with an item in
the list.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../YpbtEfXC7X1llLk+@paquier.xyz/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: michael@paquier.xyz, david.g.johnston@gmail.com, devisettypraneel@gmail.com
Subject: Re: REINDEXdb performance degrading gradually PG13.4
In-Reply-To: <YpbtEfXC7X1llLk+@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox