public inbox for [email protected]
help / color / mirror / Atom feedFrom: Tom Lane <[email protected]>
To: Andres Freund <[email protected]>
Cc: Corey Huinker <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Nathan Bossart <[email protected]>
Cc: Jeff Davis <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: jian he <[email protected]>
Cc: Bruce Momjian <[email protected]>
Cc: Matthias van de Meent <[email protected]>
Cc: Magnus Hagander <[email protected]>
Cc: Stephen Frost <[email protected]>
Cc: Ashutosh Bapat <[email protected]>
Cc: Peter Smith <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: [email protected]
Subject: Re: Statistics Import and Export
Date: Thu, 06 Mar 2025 13:47:34 -0500
Message-ID: <[email protected]> (raw)
In-Reply-To: <urk5wcvgvnqzmvdvbe6b2gthhf3pphfqmbwculciy3lm4kvnay@avvgvhxo544p>
References: <rp6int5sqs5bn7g35brqz7w3t7il6tn2dgpm7tr3pev626wnc4@fi7a3enofhct>
<Z8j8HXlIIGYPA9L8@nathan>
<CADkLM=e-H5H+pY8iA58ie28PVSRJ1HBpzyy1=FWmy0Hm+s4boQ@mail.gmail.com>
<6rpmhyrtci4epuzay7y5xvd2cwdwb6zmtt6ofxrvakyawxhm7s@6grajbpr4kij>
<CADkLM=ct+i2G=Z+c_8np1+C75uvzjUK_SBsouUvgWMHRs2cnyg@mail.gmail.com>
<xe3yjkcthy7f377zjayuckxyd62z5cekgjau3j4vp6elbicson@cx2uiilwmxyy>
<CADkLM=f1n2_Vomq0gKab7xdxDHmJGgn=DE48P8fzQOp3Mrs1Qg@mail.gmail.com>
<pbjbvxnq36xo2ooufcnlfqsqnxuvggbqxbx3myjtswokshfjbl@hfihx57z4j4t>
<CA+TgmoYGuEC=E4TbomRyqO8+Uav=90G0dYXw=rA1d5UmRB76Yg@mail.gmail.com>
<CADkLM=dTTf9tonKsQnDCDp5oyODE2mFV7K6nFuCp84QE3GBWuQ@mail.gmail.com>
<urk5wcvgvnqzmvdvbe6b2gthhf3pphfqmbwculciy3lm4kvnay@avvgvhxo544p>
Andres Freund <[email protected]> writes:
> And in contrast to analyzing the database in parallel, the pg_dump/restore
> work to restore stats afaict happens single-threaded for each database.
In principle we should be able to do stats dump/restore parallelized
just as we do for data. There are some stumbling blocks in the way
of that:
1. pg_upgrade has made a policy judgement to apply parallelism across
databases not within a database, ie it will launch concurrent dump/
restore tasks in different DBs but not authorize any one of them to
eat multiple CPUs. That needs to be re-thought probably, as I think
that decision dates to before we had useful parallelism in pg_dump and
pg_restore. I wonder if we could just rip out pg_upgrade's support
for DB-level parallelism, which is not terribly pretty anyway, and
simply pass the -j switch straight to pg_dump and pg_restore.
2. pg_restore should already be able to perform stats restores in
parallel (if authorized to use multiple threads), but I'm less clear
on whether that works right now for pg_dump.
3. Also, parallel restore depends critically on the TOC entries'
dependencies being sane, and right now I do not think they are.
I looked at "pg_restore -l -v" output for the regression DB, and it
seems like it's not taking care to ensure that table/MV data is loaded
before the table/MV's stats. (Maybe that accounts for some of the
complaints we've seen about stats getting mangled??)
> I think the stats need to be handled much more like we handle the actual table
> data, which are obviously *not* stored in memory for the whole run of pg_dump.
+1
regards, tom lane
view thread (170+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Statistics Import and Export
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox