Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xQqy0-00000000RLS-4A8n for pgsql-bugs@arkaria.postgresql.org; Tue, 10 Nov 2026 18:53:14 +0000 Received: from localhost ([127.0.0.1]:57726 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xQqxz-00000006KEE-2cL3 for pgsql-bugs@arkaria.postgresql.org; Tue, 10 Nov 2026 18:53:11 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]:39022) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xQqxz-00000006KE5-1DWe for pgsql-bugs@lists.postgresql.org; Tue, 10 Nov 2026 18:53:11 +0000 Received: from sss.pgh.pa.us ([68.162.161.243]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xQqxx-00000000HYT-0HnA for pgsql-bugs@lists.postgresql.org; Tue, 10 Nov 2026 18:53:11 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.18.1/8.18.1) with ESMTP id 6AAIr1cG2925332; Tue, 10 Nov 2026 13:53:01 -0500 DKIM-Filter: OpenDKIM Filter v2.11.0 sss.pgh.pa.us 6AAIr1cG2925332 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sss.pgh.pa.us; s=default; t=1794336781; bh=k2Ev42/ctEdxYbUaBnMWTTo+gFrcoU8CmYcsSHKQtzo=; h=From:To:cc:Subject:In-reply-to:References:Date:From; b=Nx8g+RwXzR1iQrTHOM/ZecfmGQVBSgnvZj/l7GNnjwij35Ou03LrjElMItNcaLTIb p95eMSZ0zEZuxZFr9vUXRQJwBsBXxzPDW0n0s3n6ahLqWoP9yerOKG2JK7QkLN6l4B EyEF2N4SA1DGP4hugHNRwk2Alb1W3q82hRkVtc9k= From: Tom Lane To: Michael Paquier cc: shihao zhong , Atisss hh <125atishprak@gmail.com>, pgsql-bugs@lists.postgresql.org Subject: Re: BUG #19849: queries fail after ANALYZE with expression stats on arrays In-reply-to: References: <19849-38b8bd2fea807154@postgresql.org> <2492770.1794253395@sss.pgh.pa.us> Comments: In-reply-to Michael Paquier message dated "Tue, 10 Nov 2026 15:10:42 +0900" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0" Content-ID: <2925253.1794336712.0@sss.pgh.pa.us> Date: Tue, 10 Nov 2026 13:53:01 -0500 Message-ID: <2925331.1794336781@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <2925253.1794336712.1@sss.pgh.pa.us> Michael Paquier writes: > These two could also be passed down as arguments of your new > fill_attstats_values(), filling them within the routine instead of > letting the two callers do it. > + * values[] and nulls[] must have room for Natts_pg_statistic entries. All > One thing that always stresses me with this kind of assumption is that > an error means an out-of-bound write, and a CVE. Here, we could also > use a palloc0'ed area worth Natts_pg_statistic, instead of expecting > the caller to do the right thing. The ANALYZE and expression stats > code paths are not performance-critical. Or ubsan would just catch > any problems? Yeah, I didn't love this particular refactorization either. I think it'd be nicer to have the values[] and nulls[] arrays totally inside the shared function and do the tuple-building within it, as attached. One thing that bugs me slightly about this is that it seems weird to be passing starelid and stainherit explicitly while digging staattnum out of the VacAttrStats struct. It works, because examine_expression is careful to set stats->tupattnum = InvalidAttrNumber, but maybe it would be cleaner to pass all three parts of the primary key as explicit arguments. Does anyone else feel that way? regards, tom lane ------- =_aaaaaaaaaa0 Content-Type: text/x-diff; name*0="v4-0001-Store-empty-stavalues-when-needed-for-expression-.p"; name*1="atch"; charset="us-ascii" Content-ID: <2925253.1794336712.2@sss.pgh.pa.us> Content-Description: v4-0001-Store-empty-stavalues-when-needed-for-expression-.patch Content-Transfer-Encoding: quoted-printable =46rom 32f2fe04e43f57cdff4f0adec0af6d7bdb909e75 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Tue, 10 Nov 2026 13:43:27 -0500 Subject: [PATCH v4] Store empty stavalues when needed for expression statistics. Commit 261f89a97 changed the API contract for update_attstats() to store an empty array, not a NULL, if it's passed a nonnull stavalues pointer along with a zero values count. We missed that serialize_expr_stats() needs to be kept in sync with that, so that we now get "unexpected null value" errors if statistics on an expression hit one of the cases where the statistics collection code requests that. Rather than just adjusting serialize_expr_stats() to match, let's avoid having duplicated code at all, by factoring out a routine to build a pg_statistic tuple from a VacAttrStats structure. Bug: #19849 Reported-by: Atish Prakash <125atishprak@gmail.com> Author: Atish Prakash <125atishprak@gmail.com> Co-authored-by: Shihao Zhong Co-authored-by: Tom Lane Discussion: https://postgr.es/m/19849-38b8bd2fea807154@postgresql.org Backpatch-through: 19 --- src/backend/commands/analyze.c | 201 ++++++++++++++---------- src/backend/statistics/extended_stats.c | 84 +--------- src/include/commands/vacuum.h | 4 + 3 files changed, 122 insertions(+), 167 deletions(-) diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze= .c index 518d7526255..01efa78cf76 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -1713,6 +1713,115 @@ acquire_inherited_sample_rows(Relation onerel, int= elevel, } = = +/* + * build_statistic_tuple() -- build a pg_statistic tuple from a VacAttrSt= ats + * + * relid is the Oid of the table being analyzed, inh indicates if these a= re + * inheritance-tree or table-only stats, stats contains the collected sta= ts + * including the column number. + * + * tupdesc must be a tuple descriptor for pg_statistic. oldtup is the + * existing pg_statistic tuple we intend to replace, or NULL to build + * a new pg_statistic tuple. + * + * This is exported so that extended_stats.c can use it. (In that usage, + * relid is InvalidOid, so don't trust it for anything.) + */ +HeapTuple +build_statistic_tuple(Oid relid, bool inh, + VacAttrStats *stats, + TupleDesc tupdesc, + HeapTuple oldtup) +{ + HeapTuple stup; + Datum values[Natts_pg_statistic]; + bool nulls[Natts_pg_statistic]; + bool replaces[Natts_pg_statistic]; + int i, + k; + + for (i =3D 0; i < Natts_pg_statistic; ++i) + { + nulls[i] =3D false; + replaces[i] =3D true; + } + + values[Anum_pg_statistic_starelid - 1] =3D ObjectIdGetDatum(relid); + values[Anum_pg_statistic_staattnum - 1] =3D Int16GetDatum(stats->tupattn= um); + values[Anum_pg_statistic_stainherit - 1] =3D BoolGetDatum(inh); + values[Anum_pg_statistic_stanullfrac - 1] =3D Float4GetDatum(stats->stan= ullfrac); + values[Anum_pg_statistic_stawidth - 1] =3D Int32GetDatum(stats->stawidth= ); + values[Anum_pg_statistic_stadistinct - 1] =3D Float4GetDatum(stats->stad= istinct); + i =3D Anum_pg_statistic_stakind1 - 1; + for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) + { + values[i++] =3D Int16GetDatum(stats->stakind[k]); /* stakindN */ + } + i =3D Anum_pg_statistic_staop1 - 1; + for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) + { + values[i++] =3D ObjectIdGetDatum(stats->staop[k]); /* staopN */ + } + i =3D Anum_pg_statistic_stacoll1 - 1; + for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) + { + values[i++] =3D ObjectIdGetDatum(stats->stacoll[k]); /* stacollN */ + } + i =3D Anum_pg_statistic_stanumbers1 - 1; + for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) + { + if (stats->stanumbers[k] !=3D NULL) + { + int nnum =3D stats->numnumbers[k]; + Datum *numdatums =3D palloc_array(Datum, nnum); + ArrayType *arry; + + for (int n =3D 0; n < nnum; n++) + numdatums[n] =3D Float4GetDatum(stats->stanumbers[k][n]); + arry =3D construct_array_builtin(numdatums, nnum, FLOAT4OID); + values[i++] =3D PointerGetDatum(arry); /* stanumbersN */ + } + else + { + nulls[i] =3D true; + values[i++] =3D (Datum) 0; + } + } + i =3D Anum_pg_statistic_stavalues1 - 1; + for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) + { + if (stats->stavalues[k] !=3D NULL) + { + ArrayType *arry; + + arry =3D construct_array(stats->stavalues[k], + stats->numvalues[k], + stats->statypid[k], + stats->statyplen[k], + stats->statypbyval[k], + stats->statypalign[k]); + values[i++] =3D PointerGetDatum(arry); /* stavaluesN */ + } + else + { + nulls[i] =3D true; + values[i++] =3D (Datum) 0; + } + } + + /* Build a replacement or all-new tuple */ + if (HeapTupleIsValid(oldtup)) + stup =3D heap_modify_tuple(oldtup, + tupdesc, + values, + nulls, + replaces); + else + stup =3D heap_form_tuple(tupdesc, values, nulls); + + return stup; +} + /* * update_attstats() -- update attribute statistics for one relation * @@ -1752,88 +1861,14 @@ update_attstats(Oid relid, bool inh, int natts, Va= cAttrStats **vacattrstats) VacAttrStats *stats =3D vacattrstats[attno]; HeapTuple stup, oldtup; - int i, - k, - n; - Datum values[Natts_pg_statistic]; - bool nulls[Natts_pg_statistic]; - bool replaces[Natts_pg_statistic]; = /* Ignore attr if we weren't able to collect stats */ if (!stats->stats_valid) continue; = - /* - * Construct a new pg_statistic tuple - */ - for (i =3D 0; i < Natts_pg_statistic; ++i) - { - nulls[i] =3D false; - replaces[i] =3D true; - } - - values[Anum_pg_statistic_starelid - 1] =3D ObjectIdGetDatum(relid); - values[Anum_pg_statistic_staattnum - 1] =3D Int16GetDatum(stats->tupatt= num); - values[Anum_pg_statistic_stainherit - 1] =3D BoolGetDatum(inh); - values[Anum_pg_statistic_stanullfrac - 1] =3D Float4GetDatum(stats->sta= nullfrac); - values[Anum_pg_statistic_stawidth - 1] =3D Int32GetDatum(stats->stawidt= h); - values[Anum_pg_statistic_stadistinct - 1] =3D Float4GetDatum(stats->sta= distinct); - i =3D Anum_pg_statistic_stakind1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D Int16GetDatum(stats->stakind[k]); /* stakindN */ - } - i =3D Anum_pg_statistic_staop1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D ObjectIdGetDatum(stats->staop[k]); /* staopN */ - } - i =3D Anum_pg_statistic_stacoll1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D ObjectIdGetDatum(stats->stacoll[k]); /* stacollN */ - } - i =3D Anum_pg_statistic_stanumbers1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - if (stats->stanumbers[k] !=3D NULL) - { - int nnum =3D stats->numnumbers[k]; - Datum *numdatums =3D palloc_array(Datum, nnum); - ArrayType *arry; - - for (n =3D 0; n < nnum; n++) - numdatums[n] =3D Float4GetDatum(stats->stanumbers[k][n]); - arry =3D construct_array_builtin(numdatums, nnum, FLOAT4OID); - values[i++] =3D PointerGetDatum(arry); /* stanumbersN */ - } - else - { - nulls[i] =3D true; - values[i++] =3D (Datum) 0; - } - } - i =3D Anum_pg_statistic_stavalues1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - if (stats->stavalues[k] !=3D NULL) - { - ArrayType *arry; - - arry =3D construct_array(stats->stavalues[k], - stats->numvalues[k], - stats->statypid[k], - stats->statyplen[k], - stats->statypbyval[k], - stats->statypalign[k]); - values[i++] =3D PointerGetDatum(arry); /* stavaluesN */ - } - else - { - nulls[i] =3D true; - values[i++] =3D (Datum) 0; - } - } + /* Open index information when we know we need it */ + if (indstate =3D=3D NULL) + indstate =3D CatalogOpenIndexes(sd); = /* Is there already a pg_statistic tuple for this attribute? */ oldtup =3D SearchSysCache3(STATRELATTINH, @@ -1841,25 +1876,19 @@ update_attstats(Oid relid, bool inh, int natts, Va= cAttrStats **vacattrstats) Int16GetDatum(stats->tupattnum), BoolGetDatum(inh)); = - /* Open index information when we know we need it */ - if (indstate =3D=3D NULL) - indstate =3D CatalogOpenIndexes(sd); + /* Build the new or replacement tuple */ + stup =3D build_statistic_tuple(relid, inh, stats, + RelationGetDescr(sd), oldtup); = if (HeapTupleIsValid(oldtup)) { /* Yes, replace it */ - stup =3D heap_modify_tuple(oldtup, - RelationGetDescr(sd), - values, - nulls, - replaces); ReleaseSysCache(oldtup); CatalogTupleUpdateWithInfo(sd, &stup->t_self, stup, indstate); } else { /* No, insert new tuple */ - stup =3D heap_form_tuple(RelationGetDescr(sd), values, nulls); CatalogTupleInsertWithInfo(sd, stup, indstate); } = diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statist= ics/extended_stats.c index b66a49a6fb5..da5e7264d25 100644 --- a/src/backend/statistics/extended_stats.c +++ b/src/backend/statistics/extended_stats.c @@ -2333,14 +2333,13 @@ build_expr_data(List *exprs, int stattarget) return exprdata; } = -/* form an array of pg_statistic rows (per update_attstats) */ +/* form an array of pg_statistic rows, one per expression */ static Datum serialize_expr_stats(AnlExprData *exprdata, int nexprs) { int exprno; Oid typOid; Relation sd; - ArrayBuildState *astate =3D NULL; = sd =3D table_open(StatisticRelationId, RowExclusiveLock); @@ -2355,12 +2354,7 @@ serialize_expr_stats(AnlExprData *exprdata, int nex= prs) = for (exprno =3D 0; exprno < nexprs; exprno++) { - int i, - k; VacAttrStats *stats =3D exprdata[exprno].vacattrstat; - - Datum values[Natts_pg_statistic]; - bool nulls[Natts_pg_statistic]; HeapTuple stup; = if (!stats->stats_valid) @@ -2373,80 +2367,8 @@ serialize_expr_stats(AnlExprData *exprdata, int nex= prs) continue; } = - /* - * Construct a new pg_statistic tuple - */ - for (i =3D 0; i < Natts_pg_statistic; ++i) - { - nulls[i] =3D false; - } - - values[Anum_pg_statistic_starelid - 1] =3D ObjectIdGetDatum(InvalidOid)= ; - values[Anum_pg_statistic_staattnum - 1] =3D Int16GetDatum(InvalidAttrNu= mber); - values[Anum_pg_statistic_stainherit - 1] =3D BoolGetDatum(false); - values[Anum_pg_statistic_stanullfrac - 1] =3D Float4GetDatum(stats->sta= nullfrac); - values[Anum_pg_statistic_stawidth - 1] =3D Int32GetDatum(stats->stawidt= h); - values[Anum_pg_statistic_stadistinct - 1] =3D Float4GetDatum(stats->sta= distinct); - i =3D Anum_pg_statistic_stakind1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D Int16GetDatum(stats->stakind[k]); /* stakindN */ - } - i =3D Anum_pg_statistic_staop1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D ObjectIdGetDatum(stats->staop[k]); /* staopN */ - } - i =3D Anum_pg_statistic_stacoll1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - values[i++] =3D ObjectIdGetDatum(stats->stacoll[k]); /* stacollN */ - } - i =3D Anum_pg_statistic_stanumbers1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - int nnum =3D stats->numnumbers[k]; - - if (nnum > 0) - { - int n; - Datum *numdatums =3D palloc_array(Datum, nnum); - ArrayType *arry; - - for (n =3D 0; n < nnum; n++) - numdatums[n] =3D Float4GetDatum(stats->stanumbers[k][n]); - arry =3D construct_array_builtin(numdatums, nnum, FLOAT4OID); - values[i++] =3D PointerGetDatum(arry); /* stanumbersN */ - } - else - { - nulls[i] =3D true; - values[i++] =3D (Datum) 0; - } - } - i =3D Anum_pg_statistic_stavalues1 - 1; - for (k =3D 0; k < STATISTIC_NUM_SLOTS; k++) - { - if (stats->numvalues[k] > 0) - { - ArrayType *arry; - - arry =3D construct_array(stats->stavalues[k], - stats->numvalues[k], - stats->statypid[k], - stats->statyplen[k], - stats->statypbyval[k], - stats->statypalign[k]); - values[i++] =3D PointerGetDatum(arry); /* stavaluesN */ - } - else - { - nulls[i] =3D true; - values[i++] =3D (Datum) 0; - } - } - - stup =3D heap_form_tuple(RelationGetDescr(sd), values, nulls); + stup =3D build_statistic_tuple(InvalidOid, false, stats, + RelationGetDescr(sd), NULL); = astate =3D accumArrayResult(astate, heap_copy_tuple_as_datum(stup, RelationGetDescr(sd)), diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h index 89fa1fcd261..3bcb7113762 100644 --- a/src/include/commands/vacuum.h +++ b/src/include/commands/vacuum.h @@ -441,6 +441,10 @@ extern void analyze_rel(Oid relid, RangeVar *relation= , BufferAccessStrategy bstrategy); extern bool attribute_is_analyzable(Relation onerel, int attnum, Form_pg_= attribute attr, int *p_attstattarget); +extern HeapTuple build_statistic_tuple(Oid relid, bool inh, + VacAttrStats *stats, + TupleDesc tupdesc, + HeapTuple oldtup); extern bool std_typanalyze(VacAttrStats *stats); = /* in utils/misc/sampling.c --- duplicate of declarations in utils/sampli= ng.h */ -- = 2.52.0 ------- =_aaaaaaaaaa0--