Date: Mon, 31 Mar 2025 21:33:15 -0500
From: Nathan Bossart <nathandbossart@gmail.com>
To: Corey Huinker <corey.huinker@gmail.com>
Cc: Jeff Davis <pgsql@j-davis.com>, Robert Treat <rob@xzilla.net>,
	Robert Haas <robertmhaas@gmail.com>,
	Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>,
	Michael Paquier <michael@paquier.xyz>,
	jian he <jian.universality@gmail.com>,
	Bruce Momjian <bruce@momjian.us>,
	Matthias van de Meent <boekewurm+postgres@gmail.com>,
	Magnus Hagander <magnus@hagander.net>,
	Stephen Frost <sfrost@snowman.net>,
	Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
	Peter Smith <smithpb2250@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	alvherre@alvh.no-ip.org
Subject: Re: Statistics Import and Export
Message-ID: <Z-tQa5zsVkcCyYin@nathan>
References: <e7ed5a10113d04be3d02558fe6cff06a42a8777f.camel@j-davis.com>
 <5812bf576f181e2f94944cab485cee46f611ed3e.camel@j-davis.com>
 <CADkLM=c0dOJZUQuWzVcE2BsJSVRGcX7HkJH8wYjo+OD83WFqcg@mail.gmail.com>
 <0576fa4dbd6e62e027b41e0ef9254896540f1cb4.camel@j-davis.com>
 <CADkLM=f5Dy3RtGOq=2RH2z4xLXNTwSYCAu-d5ggzQFrb8nCsyg@mail.gmail.com>
 <CADkLM=ftC94muGKH+z1irdBUXO2+tmBMLdqDAAxcct=H+LE1Eg@mail.gmail.com>
 <CADkLM=desCuf3dVHasADvdUVRmb-5gO0mhMO5u9nzgv6i7U86Q@mail.gmail.com>
 <f248676c32cbb3a4dd0da13bc84af43760239bf2.camel@j-davis.com>
 <CADkLM=dG+8jW4UAwP69o7Q92FOXpoN8Pb9kD_Fck1_BBn+gwgQ@mail.gmail.com>
 <CADkLM=domd5+CvjKMHGbOfvSuY6J8G-x+9M2D6Ss2HamYefE9w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="z+otcOHbpmiudv4t"
Content-Disposition: inline
In-Reply-To: 
 <CADkLM=domd5+CvjKMHGbOfvSuY6J8G-x+9M2D6Ss2HamYefE9w@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/Z-tQa5zsVkcCyYin%40nathan>
Precedence: bulk


--z+otcOHbpmiudv4t
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Mon, Mar 31, 2025 at 11:11:47AM -0400, Corey Huinker wrote:
> In light of v11-0001 being committed as 4694aedf63bf, I've rebased the
> remaining patches.

I spent the day preparing these for commit.  A few notes:

* I've added a new prerequisite patch that skips the second WriteToc() call
  for custom-format dumps that do not include data.  After some testing and
  code analysis, I haven't identified any examples where this produces
  different output.  This doesn't help much on its own, but it will become
  rather important when we move the attribute statistics queries to happen
  within WriteToc() in 0002.

* I was a little worried about the correctness of 0002 for dumps that run
  the attribute statistics queries twice, but I couldn't identify any
  problems here either.

* I removed a lot of miscellaneous refactoring that seemed unnecessary for
  these patches.  Let's move that to another patch set and keep these as
  simple as possible.

* I made a small adjustment to the TOC scan restarting logic in
  fetchAttributeStats().  Specifically, we now only allow the scan to
  restart once for custom-format dumps that include data.

* While these patches help decrease pg_dump's memory footprint, I believe
  pg_restore still reads the entire TOC into memory.  That's not this patch
  set's problem, but I think it's still an important consideration for the
  bigger picture.

Regarding whether pg_dump should dump statistics by default, my current
thinking is that it shouldn't, but I think we _should_ have pg_upgrade
dump/restore statistics by default because that is arguably the most
important use-case.  This is more a gut feeling than anything, so I reserve
the right to change my opinion.

My goal is to commit the attached patches on Friday morning, but of course
that is subject to change based on any feedback or objections that emerge
in the meantime.

-- 
nathan

--z+otcOHbpmiudv4t
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
	filename="v12n-0001-Skip-second-WriteToc-for-custom-format-dumps-wi.patch"