Feedback-ID: i0fe9450f:Fastmail
Date: Thu, 23 Oct 2025 08:46:27 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Corey Huinker <corey.huinker@gmail.com>
Cc: jian he <jian.universality@gmail.com>, Tomas Vondra <tomas@vondra.me>,
	pgsql-hackers@lists.postgresql.org, tgl@sss.pgh.pa.us
Subject: Re: Extended Statistics set/restore/clear functions.
Message-ID: <aPls00-bu3fPcuk-@paquier.xyz>
References: 
 <CADkLM=ekXTzxG7_-Rd7qR0Gwrq8o3kG7oSkAJRTUNSqXi0KTnA@mail.gmail.com>
 <CADkLM=dnFKZMAo7MwqD2X6JjiiLCoVXHmszqtZp8sycYmoCcMQ@mail.gmail.com>
 <CADkLM=cuVMUNsv+C7dGUrc8zF2c4C_LXBcPCtb_zHR0z+fmb1A@mail.gmail.com>
 <CADkLM=eM0VA-muiu+JMM4-B8eL5ZscPeGtTxh7UtmM+iVO7C=A@mail.gmail.com>
 <CADkLM=daHJhUf-sO1iD=UTaymbBVEgEW4Sxm8PtHWWDEGWW9jQ@mail.gmail.com>
 <CADkLM=d_=STa4RHWHk7AGZ1RAZYKMtJxGMsw0Mrdb8s6CHdvbw@mail.gmail.com>
 <CADkLM=f2B8cDpx3V2bZW07GN5JHaP7kFe7G0cmJMSPVUQ4eHhQ@mail.gmail.com>
 <CADkLM=cWJTOnhC4jbKUvFgjMd=Hc252FXbampsjuYekBvmMVAg@mail.gmail.com>
 <aPcetUfI2NJDqYxZ@paquier.xyz>
 <CADkLM=efY2h8Bio=bE8O2Tnd=WUdyKDL1Zrtz9h+at16E4HVUA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="nrslA7d3CfuiW471"
Content-Disposition: inline
In-Reply-To: 
 <CADkLM=efY2h8Bio=bE8O2Tnd=WUdyKDL1Zrtz9h+at16E4HVUA@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/aPls00-bu3fPcuk-%40paquier.xyz>
Precedence: bulk


--nrslA7d3CfuiW471
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Oct 22, 2025 at 02:55:31PM +0300, Corey Huinker wrote:
>> Do you have some numbers regarding the increase in size this generates
>> for the catalogs?
>=20
> Sorry, I don't understand. There shouldn't be any increase inside the
> catalogs as the internal storage of the datatypes hasn't changed, so I can
> only conclude that you're referring to something else.

The new format meant more characters, perhaps I've just missed
something while quickly testing the patch..  Anyway, that's OK at this
stage.

> The equivalent structures in attribute_stats.c will need documenting too.

Right.  This sounds like a separate patch to me, impacting HEAD.

> Right now we have a situation where the vast majority of databases can
> carry forward all of their stats via pg_upgrade, except for those databas=
es
> that have extended stats. The trouble is, most customers don't know if
> their database uses extended statistics or not, and those that do are in
> for some bad query plans if they haven't run vacuumdb --missing-stats-onl=
y.
> Explaining that to customers is complicated, especially when most of them
> do not know what extended stats are, let alone whether they have them. It
> would be a lot simpler to just say "all stats are carried over on upgrade=
",
> and vacuumdb becomes unnecessary, making upgrades one step simpler as wel=
l.

Okay.

> Given that, I think that the admittedly ugly transformation is worth it,
> and sequestering it inside pg_dump is the smallest footprint it can have.
> Earlier in this thread I posted some functions that did the translation
> from the existing formats to the proposed new formats. We could include
> those as new system functions, and that would make the dump code very
> simple. Having said that, I don't know that there would be use for those
> functions except inside pg_dump, hence the decision to do the transforms
> right in the dump query.

I'd prefer the new format.  One killer pushing in favor of the new
format that you are making upthread in favor of is that it makes much
easier the viewing, editing and injecting of these stats.  It's the
part of the patch where we would need Tomas' input on the matter
before deciding anything, I guess, as primary author of the original
facilities.  My view of the problem is just one opinion.

> If the format translation is a barrier to fetching existing extended stat=
s,
> then I'd be more inclined to keep the existing pg_ndistinct and
> pg_dependencies data formats as they are now.

Not necessarily, it can be possible to also take that in multiple
steps rather than a single one:
- First do the basics in v19 with the new format.
- Raise the bar to older versions.
--
Michael

--nrslA7d3CfuiW471
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmj5bNMACgkQnvQgOdby
QH3pUQ//cabq0RhflF2GLZdeIGu/dM6RfccLpOCUD5pu9KB+3uwE+hsPGZIibE1m
jLcMuFXnwMLYg02CEc+eL+THxv03Hfpk6ld3WxLUOxMkS8AadKG1YYnHX1vt7rcs
pzINa9AikJnwRV9Rn+M1k1U6flzywWsGN5Gdl35uhsX7xOlzvbCwpfxkbdaiGHpR
O5h56VmXtLzeVwETs8rCMjki4BMu9HGFWvrV1OVtNYbaBWAkNv8ihoBwBSM7/qtg
LYsUQTQh90e2wc5HxFbuMJriHCffSpneZ1t17rp996Aacvl9J9PpAwF7N0uCGT5D
pfKyYACZX9CbvurXkTO5ur3wLAfGNekRS8YTB4CphF3q62/06icRLB5NEc3uHRdJ
UEXkEAmOalZWab7nE3t2H7sHF83mxTNyaTw5PorfTnnZ7Jj3uCygjsi/6o6sR/r0
uTZ7i8tUz7HgPlfh4XJXBpa6kaym0mje7xJjgmWyBLJ4FrlowaKB0Icad1UAe0Zq
vD2to0rPX4pV5yaiwcUKfXtzQI6kKL2WIJbkOeHqETNShLcER2I1OOvLknrDDlf+
slEtmg0u6olKMYpFFUjstMZnGLfJj2Oyxvi7+nn1yhT1BW4C0cVE1xzL3+eq7VQQ
vuKiiBmW/c0ZYfxTF+6zdAU0LtcEGGamv6OAZJiWlyrlR3IZECs=
=xzEd
-----END PGP SIGNATURE-----

--nrslA7d3CfuiW471--