Message-ID: <88af35fe0ff24cc4e0700b841aa60a0865f11648.camel@j-davis.com>
Subject: Re: Statistics Import and Export
From: Jeff Davis <pgsql@j-davis.com>
To: Nathan Bossart <nathandbossart@gmail.com>, Corey Huinker
	 <corey.huinker@gmail.com>
Cc: Robert Treat <rob@xzilla.net>, Robert Haas <robertmhaas@gmail.com>, 
 Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Michael
 Paquier <michael@paquier.xyz>,  jian he <jian.universality@gmail.com>,
 Bruce Momjian <bruce@momjian.us>, Matthias van de Meent
 <boekewurm+postgres@gmail.com>, Magnus Hagander <magnus@hagander.net>, 
 Stephen Frost <sfrost@snowman.net>, Ashutosh Bapat
 <ashutosh.bapat.oss@gmail.com>, Peter Smith <smithpb2250@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, 
 alvherre@alvh.no-ip.org
Date: Tue, 01 Apr 2025 15:05:59 -0700
In-Reply-To: <Z-wz_sZdceGJ-fuI@nathan>
References: 
	<CADkLM=c0dOJZUQuWzVcE2BsJSVRGcX7HkJH8wYjo+OD83WFqcg@mail.gmail.com>
	 <0576fa4dbd6e62e027b41e0ef9254896540f1cb4.camel@j-davis.com>
	 <CADkLM=f5Dy3RtGOq=2RH2z4xLXNTwSYCAu-d5ggzQFrb8nCsyg@mail.gmail.com>
	 <CADkLM=ftC94muGKH+z1irdBUXO2+tmBMLdqDAAxcct=H+LE1Eg@mail.gmail.com>
	 <CADkLM=desCuf3dVHasADvdUVRmb-5gO0mhMO5u9nzgv6i7U86Q@mail.gmail.com>
	 <f248676c32cbb3a4dd0da13bc84af43760239bf2.camel@j-davis.com>
	 <CADkLM=dG+8jW4UAwP69o7Q92FOXpoN8Pb9kD_Fck1_BBn+gwgQ@mail.gmail.com>
	 <CADkLM=domd5+CvjKMHGbOfvSuY6J8G-x+9M2D6Ss2HamYefE9w@mail.gmail.com>
	 <Z-tQa5zsVkcCyYin@nathan> <Z-wubg4vnItKnOCj@nathan>
	 <Z-wz_sZdceGJ-fuI@nathan>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/88af35fe0ff24cc4e0700b841aa60a0865f11648.camel%40j-davis.com>
Precedence: bulk

On Tue, 2025-04-01 at 13:44 -0500, Nathan Bossart wrote:
> Apologies for the noise.=C2=A0 I noticed one more way to simplify 0002.=
=C2=A0
> As
> before, there should be no functional differences.


To restate the problem: one of the problems being solved here is that
the existing code for custom-format dumps calls WriteToc twice. That
was not a big problem before this patch, when the contents of the
entries was easily accessible in memory. But the point of 0002 is to
avoid keeping all of the stats in memory at once, because that causes
bloat; and instead to query it on demand.

In theory, we could fix the pre-existing code by making the second pass
able to jump over the other contents of the entry and just update the
data offsets. But that seems invasive, at least to do it properly.

0001 sidesteps the problem by skipping the second pass if data's not
being dumped (because there are no offsets that need updating). The
worst case is when there are a lot of objects with a small amount of
data. But that's a worst case for stats in general, so I don't think
that needs to be solved here.

Issuing the stats queries twice is not great, though. If there's any
non-deterministic output in the query, that could lead to strangeness.
How bad can that be? If the results change in some way that looks
benign, but changes the length of the definition string, can it lead to
corruption of a ToC entry? I'm not saying there's a problem, but trying
to understand the risk of future problems.

For 0003, it makes an assumption about the way the scan happens in
WriteToc(). Can you add some additional sanity checks to verify that
something doesn't happen in a different order than we expect?

Also, why do we need the clause "WHERE s.tablename =3D ANY($2)"? Isn't
that already implied by "JOIN unnest($1, $2) ... s.tablename =3D
u.tablename"?

Regards,
	Jeff Davis