pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: Corey Huinker <corey.huinker@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org
Cc: 125atishprak@gmail.com
Subject: Re: BUG #19857: Stats restore rejects empty arrays, losing range and array stats
Date: Wed, 11 Nov 2026 16:41:49 +0900
Message-ID: <avQcPMvpFykeUGli@paquier.xyz> (raw)
In-Reply-To: <CADkLM=fo1+1LwJo2Jbi3S-MS--4257Q9gA9Pt6tWhwFzzPh8Sg@mail.gmail.com>
References: <19857-6e808a09ff4b75d2@postgresql.org>
	<CADkLM=dgnnbdMF3T4Him9O+h+-_MArPzZ13SMm4g8LpAXAqmeQ@mail.gmail.com>
	<CADkLM=eyZKWTwFYh=Yyi-zqQwmkdZyVJaUD0tijoBZr=OTkFeQ@mail.gmail.com>
	<CADkLM=fo1+1LwJo2Jbi3S-MS--4257Q9gA9Pt6tWhwFzzPh8Sg@mail.gmail.com>

On Tue, Nov 10, 2026 at 08:50:39PM -0500, Corey Huinker wrote:
> Same patch, commit message updated to have both discussion threads.

 	arr = DatumGetArrayTypeP(PG_GETARG_DATUM(argnum));
 
-	if (ARR_NDIM(arr) != 1)
+	if (ARR_NDIM(arr) > 1)
 	{
 		ereport(WARNING,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),

This change would trigger a CVE, I suspect..  Under valgrind, and for
example these sequences:
CREATE TABLE t (a int);
-- most_common_freqs
SELECT pg_restore_attribute_stats(
    'schemaname', 'public', 'relname', 't', 'attname', 'a',
    'inherited', false,
    'most_common_vals', '{}'::text,
    'most_common_freqs', '{}'::real[]);
EXPLAIN SELECT * FROM t WHERE a = 1;

CREATE TABLE t2 (b int[]);
-- most_common_elem_freqs
SELECT pg_restore_attribute_stats(
    'schemaname', 'public', 'relname', 't2', 'attname', 'b',
    'inherited', false,
    'most_common_elems', '{}'::text,
    'most_common_elem_freqs', '{}'::real[]);
EXPLAIN SELECT * FROM t2 WHERE b @> '{1}';
-- elem_count_histogram
SELECT pg_clear_attribute_stats('public', 't2', 'b', false);
SELECT pg_restore_attribute_stats(
    'schemaname', 'public', 'relname', 't2', 'attname', 'b',
    'inherited', false,
    'elem_count_histogram', '{}'::real[]);
EXPLAIN SELECT * FROM t2, unnest(b);

Then get_attstatsslot() does an out-of-bound read for
most_common_freqs, most_common_elem_freqs and elem_count_histogram
because an array with 0 dimension has no follow-up data as far as I
can see, giving this pattern:
==804366== Invalid read of size 4
==804366==    at 0x1A58924: get_attstatsslot (lsyscache.c:3636)
==804366==    by 0x1950B6C: var_eq_const (selfuncs.c:422)
==804366==    by 0x195063F: eqsel_internal (selfuncs.c:349)
==804366==    by 0x1950297: eqsel (selfuncs.c:302)
==804366==    by 0x1ACE474: FunctionCall4Coll (fmgr.c:1215)
==804366==    by 0x1AD0C37: OidFunctionCall4Coll (fmgr.c:1451)
==804366==    by 0x122CC7F: restriction_selectivity (plancat.c:2242)
==804366==    by 0x1063476: clause_selectivity_ext (clausesel.c:854)
==804366==    by 0x1060B84: clauselist_selectivity_ext (clausesel.c:136)
==804366==    by 0x1060AB1: clauselist_selectivity (clausesel.c:106)
==804366==    by 0x10823BD: set_baserel_size_estimates (costsize.c:5615)
==804366==    by 0x104B64A: set_plain_rel_size (allpaths.c:641)

Instead of allowing all these arrays to be empty, we should snipe at
range_length_histogram *and* most_common_elems, and only all these
two without generalizing the rule.  The other three are rejected
anyway at load, so blocking them at restore would be a consistency
move.

Perhaps we should update get_attstatsslot() as well to avoid the
over-read issue to begin with; that feels like a sloppy thing to do if
there is an empty array to process.

We'll likely need to do something similar for the extstats.  One step
at a time.  We could of course do a more relaxing approach like you
are proposing, but I cannot really get excited to let that go through
at restore time as the loads would fail even if get_attstatsslot() is
adjusted (actually this path should be adjusted anyway), and the
option-basis thing is not that invasive in the attached, at least it
seems so to me.

Attached is a cleaned-up patch.  So, thoughts or comments?
--
Michael

Attachments:

0001-Allow-empty-arrays-of-values-in-pg_restore_attribute.patch (text/plain, 9.6K) download From f6a8026a4b995e0d0dc55bd73b770ddfa4c77889 Mon Sep 17 00:00:00 2001 From: Michael Paquier <michael@paquier.xyz> Date: Wed, 11 Nov 2026 16:25:33 +0900 Subject: [PATCH] Allow empty arrays of values in pg_restore_attribute_stats() ANALYZE can store an empty array of values for range_length_histogram and most_common_elems, which pg_dump emits as '{}'. These were rejected by pg_restore_attribute_stats(), as an empty array has no dimensions, and the statistics were lost after a dump/restore or an upgrade. This commit allows empty arrays for these two cases only. Arrays stored as stanumbers are still required to be non-empty, as ANALYZE never generates them. get_attstatsslot() fails on them. Some tests are added to cover both the authorized and non-authorized patterns, in case somebody has a change of mind in the future regarding one or more of them. Reported-by: Atish Prakash <125atishprak@gmail.com> Author: Corey Huinker <corey.huinker@gmail.com> Author: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/19857-6e808a09ff4b75d2@postgresql.org Backpatch-through: 18 --- src/include/statistics/stat_utils.h | 3 +- src/backend/statistics/attribute_stats.c | 10 ++-- src/backend/statistics/stat_utils.c | 12 +++- src/test/regress/expected/stats_import.out | 67 ++++++++++++++++++++++ src/test/regress/sql/stats_import.sql | 40 +++++++++++++ 5 files changed, 123 insertions(+), 9 deletions(-) diff --git a/src/include/statistics/stat_utils.h b/src/include/statistics/stat_utils.h index 957bddc5b2b9..d160f9fbd4bd 100644 --- a/src/include/statistics/stat_utils.h +++ b/src/include/statistics/stat_utils.h @@ -57,7 +57,8 @@ extern void statatt_set_slot(Datum *values, bool *nulls, bool *replaces, Datum stavalues, bool stavalues_isnull); extern Datum statatt_build_stavalues(const char *staname, FmgrInfo *array_in, Datum d, - Oid typid, int32 typmod, bool *ok); + Oid typid, int32 typmod, bool allow_empty, + bool *ok); extern bool statatt_get_elem_type(TypeCacheEntry *basetypcache, Oid *elemtypid, Oid *elem_eq_opr); extern bool statatt_get_range_type(TypeCacheEntry *basetypcache, diff --git a/src/backend/statistics/attribute_stats.c b/src/backend/statistics/attribute_stats.c index f4bb846e88fb..48e827d16af2 100644 --- a/src/backend/statistics/attribute_stats.c +++ b/src/backend/statistics/attribute_stats.c @@ -399,7 +399,7 @@ attribute_statistics_update_internal(Oid reloid, &array_in_fn, PG_GETARG_DATUM(MOST_COMMON_VALS_ARG), atttypid, atttypmod, - &converted); + false, &converted); if (converted) { @@ -439,7 +439,7 @@ attribute_statistics_update_internal(Oid reloid, &array_in_fn, PG_GETARG_DATUM(HISTOGRAM_BOUNDS_ARG), atttypid, atttypmod, - &converted); + false, &converted); if (converted) { @@ -476,7 +476,7 @@ attribute_statistics_update_internal(Oid reloid, &array_in_fn, PG_GETARG_DATUM(MOST_COMMON_ELEMS_ARG), elemtypid, atttypmod, - &converted); + true, &converted); if (converted) { @@ -518,7 +518,7 @@ attribute_statistics_update_internal(Oid reloid, &array_in_fn, PG_GETARG_DATUM(RANGE_BOUNDS_HISTOGRAM_ARG), bounds_typid, atttypmod, - &converted); + false, &converted); if (converted && statatt_check_bounds_histogram(stavalues)) @@ -546,7 +546,7 @@ attribute_statistics_update_internal(Oid reloid, stavalues = statatt_build_stavalues("range_length_histogram", &array_in_fn, PG_GETARG_DATUM(RANGE_LENGTH_HISTOGRAM_ARG), - FLOAT8OID, 0, &converted); + FLOAT8OID, 0, true, &converted); if (converted) { diff --git a/src/backend/statistics/stat_utils.c b/src/backend/statistics/stat_utils.c index e8117ede407b..9f8da2202c7b 100644 --- a/src/backend/statistics/stat_utils.c +++ b/src/backend/statistics/stat_utils.c @@ -85,7 +85,7 @@ stats_check_arg_array(FunctionCallInfo fcinfo, { ereport(WARNING, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("argument \"%s\" must not be a multidimensional array", + errmsg("argument \"%s\" must be a non-empty one-dimensional array", arginfo[argnum].argname))); return false; } @@ -597,11 +597,16 @@ statatt_get_range_type(TypeCacheEntry *basetypcache, Oid *rangetypid) * If an error is encountered, capture it and throw a WARNING, with "ok" set * to false. If the resulting array contains NULLs, raise a WARNING and * set "ok" to false. When the operation succeeds, set "ok" to true. + * + * The array must be one-dimensional. If "allow_empty" is true, an empty + * array is also accepted. */ Datum statatt_build_stavalues(const char *staname, FmgrInfo *array_in, Datum d, Oid typid, - int32 typmod, bool *ok) + int32 typmod, bool allow_empty, bool *ok) { + int ndim; + char *s; Datum result; ErrorSaveContext escontext = {T_ErrorSaveContext}; @@ -622,7 +627,8 @@ statatt_build_stavalues(const char *staname, FmgrInfo *array_in, Datum d, Oid ty pfree(s); - if (ARR_NDIM(DatumGetArrayTypeP(result)) != 1) + ndim = ARR_NDIM(DatumGetArrayTypeP(result)); + if (ndim != 1 && !(allow_empty && ndim == 0)) { ereport(WARNING, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), diff --git a/src/test/regress/expected/stats_import.out b/src/test/regress/expected/stats_import.out index 60a0f20b488e..bc9ab2498736 100644 --- a/src/test/regress/expected/stats_import.out +++ b/src/test/regress/expected/stats_import.out @@ -4324,6 +4324,73 @@ SELECT count(*) FROM pg_statistic s 0 (1 row) +-- Empty arrays of values +-- ok: range_length_histogram +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'arange', + 'inherited', false::boolean, + 'range_length_histogram', '{}'::text, + 'range_empty_frac', '1'::real); + pg_restore_attribute_stats +---------------------------- + t +(1 row) + +-- ok: most_common_elems +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'tags', + 'inherited', false::boolean, + 'most_common_elems', '{}'::text, + 'most_common_elem_freqs', '{0.1,0.1,0}'::real[]); + pg_restore_attribute_stats +---------------------------- + t +(1 row) + +SELECT attname, range_length_histogram, range_empty_frac, + most_common_elems, most_common_elem_freqs + FROM pg_stats + WHERE schemaname = 'stats_import' + AND tablename = 'test' + AND attname IN ('arange', 'tags') + ORDER BY attname; + attname | range_length_histogram | range_empty_frac | most_common_elems | most_common_elem_freqs +---------+------------------------+------------------+-------------------+------------------------ + arange | {} | 1 | | + tags | | | {} | {0.1,0.1,0} +(2 rows) + +-- warn: most_common_vals and most_common_freqs +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'id', + 'inherited', false::boolean, + 'most_common_vals', '{}'::text, + 'most_common_freqs', '{}'::real[]); +WARNING: argument "most_common_freqs" must be a non-empty one-dimensional array + pg_restore_attribute_stats +---------------------------- + f +(1 row) + +-- warn: histogram_bounds +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'id', + 'inherited', false::boolean, + 'histogram_bounds', '{}'::text); +WARNING: "histogram_bounds" must be a one-dimensional array + pg_restore_attribute_stats +---------------------------- + f +(1 row) + DROP SCHEMA stats_import CASCADE; NOTICE: drop cascades to 39 other objects DETAIL: drop cascades to view stats_import.pg_stats_stable diff --git a/src/test/regress/sql/stats_import.sql b/src/test/regress/sql/stats_import.sql index e537a9d6f67b..6e682b83493d 100644 --- a/src/test/regress/sql/stats_import.sql +++ b/src/test/regress/sql/stats_import.sql @@ -3063,4 +3063,44 @@ SELECT count(*) FROM pg_statistic s AND a.attnum = s.staattnum AND NOT a.attisdropped); +-- Empty arrays of values +-- ok: range_length_histogram +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'arange', + 'inherited', false::boolean, + 'range_length_histogram', '{}'::text, + 'range_empty_frac', '1'::real); +-- ok: most_common_elems +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'tags', + 'inherited', false::boolean, + 'most_common_elems', '{}'::text, + 'most_common_elem_freqs', '{0.1,0.1,0}'::real[]); +SELECT attname, range_length_histogram, range_empty_frac, + most_common_elems, most_common_elem_freqs + FROM pg_stats + WHERE schemaname = 'stats_import' + AND tablename = 'test' + AND attname IN ('arange', 'tags') + ORDER BY attname; +-- warn: most_common_vals and most_common_freqs +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'id', + 'inherited', false::boolean, + 'most_common_vals', '{}'::text, + 'most_common_freqs', '{}'::real[]); +-- warn: histogram_bounds +SELECT pg_catalog.pg_restore_attribute_stats( + 'schemaname', 'stats_import', + 'relname', 'test', + 'attname', 'id', + 'inherited', false::boolean, + 'histogram_bounds', '{}'::text); + DROP SCHEMA stats_import CASCADE; -- 2.56.0
signature.asc (application/pgp-signature, 832B) download

view thread (13+ messages)  latest in thread

Message-ID: <avQcPMvpFykeUGli@paquier.xyz>
Permalink:  ../avQcPMvpFykeUGli@paquier.xyz/
Also on:    postgresql.org/message-id/avQcPMvpFykeUGli@paquier.xyz

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: michael@paquier.xyz, corey.huinker@gmail.com, 125atishprak@gmail.com
  Subject: Re: BUG #19857: Stats restore rejects empty arrays, losing range and array stats
  In-Reply-To: <avQcPMvpFykeUGli@paquier.xyz>
  References: <19857-6e808a09ff4b75d2@postgresql.org> <CADkLM=dgnnbdMF3T4Him9O+h+-_MArPzZ13SMm4g8LpAXAqmeQ@mail.gmail.com> <CADkLM=eyZKWTwFYh=Yyi-zqQwmkdZyVJaUD0tijoBZr=OTkFeQ@mail.gmail.com> <CADkLM=fo1+1LwJo2Jbi3S-MS--4257Q9gA9Pt6tWhwFzzPh8Sg@mail.gmail.com> <avQcPMvpFykeUGli@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox