public inbox for [email protected]
help / color / mirror / Atom feedFrom: Tomas Vondra <[email protected]>
To: Michael Paquier <[email protected]>
To: Corey Huinker <[email protected]>
Cc: jian he <[email protected]>
Cc: [email protected]
Cc: [email protected]
Subject: Re: Extended Statistics set/restore/clear functions.
Date: Fri, 31 Oct 2025 21:22:55 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <CADkLM=ekXTzxG7_-Rd7qR0Gwrq8o3kG7oSkAJRTUNSqXi0KTnA@mail.gmail.com>
<CADkLM=dnFKZMAo7MwqD2X6JjiiLCoVXHmszqtZp8sycYmoCcMQ@mail.gmail.com>
<CADkLM=cuVMUNsv+C7dGUrc8zF2c4C_LXBcPCtb_zHR0z+fmb1A@mail.gmail.com>
<CADkLM=eM0VA-muiu+JMM4-B8eL5ZscPeGtTxh7UtmM+iVO7C=A@mail.gmail.com>
<CADkLM=daHJhUf-sO1iD=UTaymbBVEgEW4Sxm8PtHWWDEGWW9jQ@mail.gmail.com>
<CADkLM=d_=STa4RHWHk7AGZ1RAZYKMtJxGMsw0Mrdb8s6CHdvbw@mail.gmail.com>
<CADkLM=f2B8cDpx3V2bZW07GN5JHaP7kFe7G0cmJMSPVUQ4eHhQ@mail.gmail.com>
<CADkLM=cWJTOnhC4jbKUvFgjMd=Hc252FXbampsjuYekBvmMVAg@mail.gmail.com>
<[email protected]>
<CADkLM=efY2h8Bio=bE8O2Tnd=WUdyKDL1Zrtz9h+at16E4HVUA@mail.gmail.com>
<[email protected]>
On 10/23/25 01:46, Michael Paquier wrote:
> On Wed, Oct 22, 2025 at 02:55:31PM +0300, Corey Huinker wrote:
>>> Do you have some numbers regarding the increase in size this generates
>>> for the catalogs?
>>
>> Sorry, I don't understand. There shouldn't be any increase inside the
>> catalogs as the internal storage of the datatypes hasn't changed, so I can
>> only conclude that you're referring to something else.
>
> The new format meant more characters, perhaps I've just missed
> something while quickly testing the patch.. Anyway, that's OK at this
> stage.
>
>> The equivalent structures in attribute_stats.c will need documenting too.
>
> Right. This sounds like a separate patch to me, impacting HEAD.
>
>> Right now we have a situation where the vast majority of databases can
>> carry forward all of their stats via pg_upgrade, except for those databases
>> that have extended stats. The trouble is, most customers don't know if
>> their database uses extended statistics or not, and those that do are in
>> for some bad query plans if they haven't run vacuumdb --missing-stats-only.
>> Explaining that to customers is complicated, especially when most of them
>> do not know what extended stats are, let alone whether they have them. It
>> would be a lot simpler to just say "all stats are carried over on upgrade",
>> and vacuumdb becomes unnecessary, making upgrades one step simpler as well.
>
> Okay.
>
>> Given that, I think that the admittedly ugly transformation is worth it,
>> and sequestering it inside pg_dump is the smallest footprint it can have.
>> Earlier in this thread I posted some functions that did the translation
>> from the existing formats to the proposed new formats. We could include
>> those as new system functions, and that would make the dump code very
>> simple. Having said that, I don't know that there would be use for those
>> functions except inside pg_dump, hence the decision to do the transforms
>> right in the dump query.
>
> I'd prefer the new format. One killer pushing in favor of the new
> format that you are making upthread in favor of is that it makes much
> easier the viewing, editing and injecting of these stats. It's the
> part of the patch where we would need Tomas' input on the matter
> before deciding anything, I guess, as primary author of the original
> facilities. My view of the problem is just one opinion.
>
Sorry for not paying much attention to this thread ...
My opinion is that we should both use the new format and keep the
pg_dump code to allow upgrading from older pre-19 versions.
There really is nothing special about the current format - I should have
used JSON (or any other established format) from the beginning. But I
only saw that as human-readable version of ephemeral data, it didn't
occur to me we'll use this to export/import stats cross versions. So if
we need to adjust that to make new use cases more convenient, let's bite
the bullet now.
If doing both is too complex / ugly, I think the pg_upgrade capability
is more valuable. I'd rather keep the old, less convenient format to
have pg_upgrade support for all versions.
Otherwise users may not benefit from this pg_upgrade feature for a
couple more years. Plenty of users delay upgrading until the EOL gets
close, and so might be unable to dump/restore extended stats for the
next ~5 years.
regards
--
Tomas Vondra
view thread (91+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Extended Statistics set/restore/clear functions.
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox