Message-ID: <6f02786f4f303fe279963d1546e105762072a187.camel@j-davis.com>
Subject: Re: Statistics Import and Export
From: Jeff Davis <pgsql@j-davis.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Corey Huinker <corey.huinker@gmail.com>, Robert Treat <rob@xzilla.net>, 
 Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
 Tom Lane <tgl@sss.pgh.pa.us>,  Michael Paquier <michael@paquier.xyz>, jian
 he <jian.universality@gmail.com>, Bruce Momjian <bruce@momjian.us>, 
 Matthias van de Meent <boekewurm+postgres@gmail.com>, Magnus Hagander
 <magnus@hagander.net>, Stephen Frost <sfrost@snowman.net>,  Ashutosh Bapat
 <ashutosh.bapat.oss@gmail.com>, Peter Smith <smithpb2250@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 alvherre@alvh.no-ip.org
Date: Wed, 02 Apr 2025 16:36:58 -0700
In-Reply-To: <Z-ytSGUZAQNIM9GJ@nathan>
References: 
	<CADkLM=f5Dy3RtGOq=2RH2z4xLXNTwSYCAu-d5ggzQFrb8nCsyg@mail.gmail.com>
	 <CADkLM=ftC94muGKH+z1irdBUXO2+tmBMLdqDAAxcct=H+LE1Eg@mail.gmail.com>
	 <CADkLM=desCuf3dVHasADvdUVRmb-5gO0mhMO5u9nzgv6i7U86Q@mail.gmail.com>
	 <f248676c32cbb3a4dd0da13bc84af43760239bf2.camel@j-davis.com>
	 <CADkLM=dG+8jW4UAwP69o7Q92FOXpoN8Pb9kD_Fck1_BBn+gwgQ@mail.gmail.com>
	 <CADkLM=domd5+CvjKMHGbOfvSuY6J8G-x+9M2D6Ss2HamYefE9w@mail.gmail.com>
	 <Z-tQa5zsVkcCyYin@nathan> <Z-wubg4vnItKnOCj@nathan>
	 <Z-wz_sZdceGJ-fuI@nathan>
	 <88af35fe0ff24cc4e0700b841aa60a0865f11648.camel@j-davis.com>
	 <Z-ytSGUZAQNIM9GJ@nathan>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/6f02786f4f303fe279963d1546e105762072a187.camel%40j-davis.com>
Precedence: bulk

On Tue, 2025-04-01 at 22:21 -0500, Nathan Bossart wrote:
> It certainly feels risky.=C2=A0 I was able to avoid executing the queries
> twice
> in all cases by saving the definition length in the TOC entry and
> skipping
> that many bytes the second time round.

Another idea that was under-discussed is whether the stats commands
should be in the TOC at all, or if they should be written as data
chunks.

Being in the TOC creates these issues with rewriting the TOC. Also, the
stats can be fairly large, especially for a wide table with a high
stats target, so the stats commands can increase the size of the TOC by
a lot.

But putting them in the data area doesn't seem quite right either,
because the data is just data, whereas the stats are a list of SQL
commands ("SELECT pg_restore_relation_stats(...); ..."). Also, if we
went down that road, we'd have to consider parallelism, which might
defeat the batching work that we're trying to do.

Regards,
	Jeff Davis