Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vYelw-00AIHW-2U for pgsql-hackers@arkaria.postgresql.org; Thu, 25 Dec 2025 06:24:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vYelv-006psk-2F for pgsql-hackers@arkaria.postgresql.org; Thu, 25 Dec 2025 06:24:28 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vYelu-006psc-0t for pgsql-hackers@lists.postgresql.org; Thu, 25 Dec 2025 06:24:28 +0000 Received: from fhigh-b5-smtp.messagingengine.com ([202.12.124.156]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vYelq-002VsM-2D for pgsql-hackers@lists.postgresql.org; Thu, 25 Dec 2025 06:24:26 +0000 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 40B837A00D5; Thu, 25 Dec 2025 01:24:21 -0500 (EST) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Thu, 25 Dec 2025 01:24:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1766643861; x=1766730261; bh=SatbCHkxyA XrNuymhNBgBejhG3u6vek4SWQmN9cgG/Q=; b=dKB4BRSPXSVULZ9XPO9iPa/sMF EhLZ7fhj15taaLhXr6AIPZU49Fhcf5YeIYi1ru6jDzwaJnSEJW1KGsR2sLmFiqpF KM2rHXMtiErNI0XenXq28CDwTRCjzCkAqGhK+mnv/fVZemmHj511d45eU+nLxA0r tRXESzUqp6hLA+sKMUWlWlDKPLdFtJ+krpcClE7H4yVWSL5o7uZYBSOyTnsTqH2c x37tHHbK8Q8tsButJbiqVyYnhPHAzS6U79sWyD0/JCCPQnLvs1KxDKGk+GszJkdn VXHnTOTECi+kq15/Oi4pxjVWNzhUJkptswih7xZsoJ6iXV4o35jetOoHft3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1766643861; x=1766730261; bh=SatbCHkxyAXrNuymhNBgBejhG3u6vek4SWQ mN9cgG/Q=; b=JHbnpNk4BW8zBBXb3KQODAumVo8cbdf6iep3FqEi0TkdHq99euj UPJfqz+GkbQTuccJ1r3/scrgOF0xK+Va9gs2FaqvMy97vhkkW9S9LnyM0LzA1eTn Sw560di7q/DAV3VNCCKpQinhIhpYznv8SR7/DB2VzqqFrUz2WErnec8zqAFTxj8U 7AVCRfv7GBemJt10oXElp5j5lYYRIZ6f2/I/CJe+p2saLhwA98kWUavNQjKIrlE5 Jkaf7kZefzLqatFyk0R/GizOzeDA3qIwwqrfWHvvEIh4VvU/BuAfgtTkRbNdIMhJ 8MYm+25gIh3vsxGxbNxaPS6uBC7rHXcsaBg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefgedrtddtgdeigeekhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlh cuvffnffculdejtddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomhepofhitghhrggvlhcurfgrqhhuihgvrhcuoehmihgthhgrvghlsehprghquh hivghrrdighiiiqeenucggtffrrghtthgvrhhnpeetleeifedufffhhfdtteelgeeggeff hfekueevteeigfduudevudetgfegiedvjeenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehmihgthhgrvghlsehprghquhhivghrrdighiiipdhn sggprhgtphhtthhopeehpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegtohhrvg ihrdhhuhhinhhkvghrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepthhglhesshhsshdr phhghhdrphgrrdhushdprhgtphhtthhopehjihgrnhdruhhnihhvvghrshgrlhhithihse hgmhgrihhlrdgtohhmpdhrtghpthhtohepthhomhgrshesvhhonhgurhgrrdhmvgdprhgt phhtthhopehpghhsqhhlqdhhrggtkhgvrhhssehlihhsthhsrdhpohhsthhgrhgvshhqlh drohhrgh X-ME-Proxy: Feedback-ID: i0fe9450f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 25 Dec 2025 01:24:16 -0500 (EST) Date: Thu, 25 Dec 2025 15:23:58 +0900 From: Michael Paquier To: Corey Huinker Cc: Tom Lane , jian he , Tomas Vondra , pgsql-hackers@lists.postgresql.org Subject: Re: Extended Statistics set/restore/clear functions. Message-ID: References: <1341064.1764895052@sss.pgh.pa.us> <1353179.1764901790@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qEAULHxEpVnoWmAO" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --qEAULHxEpVnoWmAO Content-Type: multipart/mixed; boundary="mcHHJuWZHCX7IVwX" Content-Disposition: inline --mcHHJuWZHCX7IVwX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Dec 11, 2025 at 04:55:52PM -0500, Corey Huinker wrote: > Disregard v21 (uncommitted changes). Everything I said about 21 applies to > 22. A couple of weeks later, I have locked a couple of hours to look at v22-0001. And after a couple of rounds of self-fixing and adjustments in the documentation added, the result looks OK so I have applied it. Attached are the remaining pieces, labelled v23. -- Michael --mcHHJuWZHCX7IVwX Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v23-0001-Add-extended-statistics-support-functions.patch" Content-Transfer-Encoding: quoted-printable =46rom 073ebafcaadef0afdd1444c8bfca13ae4665b68b Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Thu, 25 Dec 2025 15:20:56 +0900 Subject: [PATCH v23 1/2] Add extended statistics support functions. Add pg_restore_extended_stats() and pg_clear_extended_stats(). These functions closely mirror their relation and attribute counterparts, but for extended statistics (i.e. CREATE STATISTICS) objects. --- src/include/catalog/pg_proc.dat | 18 + .../statistics/extended_stats_internal.h | 12 + src/backend/statistics/dependencies.c | 63 + src/backend/statistics/extended_stats.c | 1205 +++++++++++++++++ src/backend/statistics/mcv.c | 144 ++ src/backend/statistics/mvdistinct.c | 62 + src/test/regress/expected/stats_import.out | 578 ++++++++ src/test/regress/sql/stats_import.sql | 364 +++++ doc/src/sgml/func/func-admin.sgml | 98 ++ 9 files changed, 2544 insertions(+) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.= dat index fd9448ec7b98..22fe1f0a9f8e 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12602,6 +12602,24 @@ proname =3D> 'gist_translate_cmptype_common', prorettype =3D> 'int2', proargtypes =3D> 'int4', prosrc =3D> 'gist_translate_cmptype_common' }, =20 +# Extended Statistics Import +{ oid =3D> '9947', + descr =3D> 'restore statistics on extended statistics object', + proname =3D> 'pg_restore_extended_stats', provolatile =3D> 'v', proisstr= ict =3D> 'f', + provariadic =3D> 'any', + proparallel =3D> 'u', prorettype =3D> 'bool', + proargtypes =3D> 'any', + proargnames =3D> '{kwargs}', + proargmodes =3D> '{v}', + prosrc =3D> 'pg_restore_extended_stats' }, +{ oid =3D> '9948', + descr =3D> 'clear statistics on extended statistics object', + proname =3D> 'pg_clear_extended_stats', provolatile =3D> 'v', proisstric= t =3D> 'f', + proparallel =3D> 'u', prorettype =3D> 'void', + proargtypes =3D> 'text text bool', + proargnames =3D> '{statistics_schemaname,statistics_name,inherited}', + prosrc =3D> 'pg_clear_extended_stats' }, + # AIO related functions { oid =3D> '6399', descr =3D> 'information about in-progress asynchronous = IOs', proname =3D> 'pg_get_aios', prorows =3D> '100', proretset =3D> 't', diff --git a/src/include/statistics/extended_stats_internal.h b/src/include= /statistics/extended_stats_internal.h index efcb7dc35461..042f07a76029 100644 --- a/src/include/statistics/extended_stats_internal.h +++ b/src/include/statistics/extended_stats_internal.h @@ -127,4 +127,16 @@ extern Selectivity mcv_clause_selectivity_or(PlannerIn= fo *root, Selectivity *overlap_basesel, Selectivity *totalsel); =20 +extern Datum import_mcvlist(HeapTuple tup, int elevel, int numattrs, + Oid *atttypids, int32 *atttypmods, Oid *atttypcolls, + int nitems, Datum *mcv_elems, bool *mcv_nulls, + bool *mcv_elem_nulls, float8 *freqs, float8 *base_freqs); +extern bool pg_ndistinct_validate_items(MVNDistinct *ndistinct, int2vector= *stxkeys, + int numexprs, int elevel); +extern void free_pg_ndistinct(MVNDistinct *ndistinct); +extern bool pg_dependencies_validate_deps(const MVDependencies *dependenci= es, + const int2vector *stxkeys, int numexprs, + int elevel); +extern void free_pg_dependencies(MVDependencies *dependencies); + #endif /* EXTENDED_STATS_INTERNAL_H */ diff --git a/src/backend/statistics/dependencies.c b/src/backend/statistics= /dependencies.c index 2aed867d5e7c..215b7b690f0f 100644 --- a/src/backend/statistics/dependencies.c +++ b/src/backend/statistics/dependencies.c @@ -1064,6 +1064,57 @@ clauselist_apply_dependencies(PlannerInfo *root, Lis= t *clauses, return s1; } =20 +/* + * Validate an MVDependencies against the extended statistics object defin= ition. + * + * Every MVDependencies must be checked to ensure that the attnums in the + * attributes list correspond to attnums/expressions defined by the + * extended statistics object. + * + * Positive attnums are attributes which must be found in the stxkeys, + * while negative attnums correspond to an expr number, so the attnum + * can't be below (0 - numexprs). + */ +bool +pg_dependencies_validate_deps(const MVDependencies *dependencies, + const int2vector *stxkeys, + int numexprs, int elevel) +{ + int attnum_expr_lowbound =3D 0 - numexprs; + + for (int i =3D 0; i < dependencies->ndeps; i++) + { + const MVDependency *dep =3D dependencies->deps[i]; + + for (int j =3D 0; j < dep->nattributes; j++) + { + AttrNumber attnum =3D dep->attributes[j]; + bool ok =3D false; + + if (attnum > 0) + { + for (int k =3D 0; k < stxkeys->dim1; k++) + if (attnum =3D=3D stxkeys->values[k]) + { + ok =3D true; + break; + } + } + else if ((attnum < 0) && (attnum >=3D attnum_expr_lowbound)) + ok =3D true; + + if (!ok) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("pg_dependencies: invalid attnum for this statistics object:= %d", attnum))); + return false; + } + } + } + return true; +} + /* * dependency_is_compatible_expression * Determines if the expression is compatible with functional dependencies @@ -1247,6 +1298,18 @@ dependency_is_compatible_expression(Node *clause, In= dex relid, List *statlist, N return false; } =20 +/* + * Free allocations of an MVNDistinct + */ +void +free_pg_dependencies(MVDependencies *dependencies) +{ + for (int i =3D 0; i < dependencies->ndeps; i++) + pfree(dependencies->deps[i]); + + pfree(dependencies); +} + /* * dependencies_clauselist_selectivity * Return the estimated selectivity of (a subset of) the given clauses diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statisti= cs/extended_stats.c index 19778b773d20..715975f36ac8 100644 --- a/src/backend/statistics/extended_stats.c +++ b/src/backend/statistics/extended_stats.c @@ -18,21 +18,28 @@ =20 #include "access/detoast.h" #include "access/genam.h" +#include "access/heapam.h" +#include "access/htup.h" #include "access/htup_details.h" #include "access/table.h" #include "catalog/indexing.h" +#include "catalog/pg_collation.h" #include "catalog/pg_statistic_ext.h" #include "catalog/pg_statistic_ext_data.h" +#include "catalog/pg_type_d.h" +#include "catalog/namespace.h" #include "commands/defrem.h" #include "commands/progress.h" #include "executor/executor.h" #include "miscadmin.h" +#include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" #include "optimizer/optimizer.h" #include "parser/parsetree.h" #include "pgstat.h" #include "postmaster/autovacuum.h" #include "statistics/extended_stats_internal.h" +#include "statistics/stat_utils.h" #include "statistics/statistics.h" #include "utils/acl.h" #include "utils/array.h" @@ -72,6 +79,84 @@ typedef struct StatExtEntry List *exprs; /* expressions */ } StatExtEntry; =20 +/* + * An index of the args for extended_statistics_update(). + */ +enum extended_stats_argnum +{ + STATSCHEMA_ARG =3D 0, + STATNAME_ARG, + INHERITED_ARG, + NDISTINCT_ARG, + DEPENDENCIES_ARG, + MOST_COMMON_VALS_ARG, + MOST_COMMON_VAL_NULLS_ARG, + MOST_COMMON_FREQS_ARG, + MOST_COMMON_BASE_FREQS_ARG, + EXPRESSIONS_ARG, + NUM_EXTENDED_STATS_ARGS +}; + +/* + * The argument names and typoids of the arguments for + * extended_statistics_update(). + */ +static struct StatsArgInfo extarginfo[] =3D +{ + [STATSCHEMA_ARG] =3D {"statistics_schemaname", TEXTOID}, + [STATNAME_ARG] =3D {"statistics_name", TEXTOID}, + [INHERITED_ARG] =3D {"inherited", BOOLOID}, + [NDISTINCT_ARG] =3D {"n_distinct", PG_NDISTINCTOID}, + [DEPENDENCIES_ARG] =3D {"dependencies", PG_DEPENDENCIESOID}, + [MOST_COMMON_VALS_ARG] =3D {"most_common_vals", TEXTARRAYOID}, + [MOST_COMMON_VAL_NULLS_ARG] =3D {"most_common_val_nulls", BOOLARRAYOID}, + [MOST_COMMON_FREQS_ARG] =3D {"most_common_freqs", FLOAT8ARRAYOID}, + [MOST_COMMON_BASE_FREQS_ARG] =3D {"most_common_base_freqs", FLOAT8ARRAYOI= D}, + [EXPRESSIONS_ARG] =3D {"exprs", TEXTARRAYOID}, + [NUM_EXTENDED_STATS_ARGS] =3D {0} +}; + +/* + * An index of the elements of the stxdexprs datum, which repeat for each + * expression in the extended statistics object. + * + * NOTE: the RANGE_LENGTH & RANGE_BOUNDS stats are not yet reflected in any + * version of pg_stat_ext_exprs. + */ +enum extended_stats_exprs_element +{ + NULL_FRAC_ELEM =3D 0, + AVG_WIDTH_ELEM, + N_DISTINCT_ELEM, + MOST_COMMON_VALS_ELEM, + MOST_COMMON_FREQS_ELEM, + HISTOGRAM_BOUNDS_ELEM, + CORRELATION_ELEM, + MOST_COMMON_ELEMS_ELEM, + MOST_COMMON_ELEM_FREQS_ELEM, + ELEM_COUNT_HISTOGRAM_ELEM, + NUM_ATTRIBUTE_STATS_ELEMS +}; + +/* + * The argument names and typoids of the repeating arguments for stxdexprs. + */ +static struct StatsArgInfo extexprarginfo[] =3D +{ + [NULL_FRAC_ELEM] =3D {"null_frac", FLOAT4OID}, + [AVG_WIDTH_ELEM] =3D {"avg_width", INT4OID}, + [N_DISTINCT_ELEM] =3D {"n_distinct", FLOAT4OID}, + [MOST_COMMON_VALS_ELEM] =3D {"most_common_vals", TEXTOID}, + [MOST_COMMON_FREQS_ELEM] =3D {"most_common_freqs", FLOAT4ARRAYOID}, + [HISTOGRAM_BOUNDS_ELEM] =3D {"histogram_bounds", TEXTOID}, + [CORRELATION_ELEM] =3D {"correlation", FLOAT4OID}, + [MOST_COMMON_ELEMS_ELEM] =3D {"most_common_elems", TEXTOID}, + [MOST_COMMON_ELEM_FREQS_ELEM] =3D {"most_common_elem_freqs", FLOAT4ARRAYO= ID}, + [ELEM_COUNT_HISTOGRAM_ELEM] =3D {"elem_count_histogram", FLOAT4ARRAYOID}, + [NUM_ATTRIBUTE_STATS_ELEMS] =3D {0} +}; + +static bool extended_statistics_update(FunctionCallInfo fcinfo); =20 static List *fetch_statentries_for_relation(Relation pg_statext, Oid relid= ); static VacAttrStats **lookup_var_attr_stats(Bitmapset *attrs, List *exprs, @@ -99,6 +184,30 @@ static StatsBuildData *make_build_data(Relation rel, St= atExtEntry *stat, int numrows, HeapTuple *rows, VacAttrStats **stats, int stattarget); =20 +static HeapTuple get_pg_statistic_ext(Relation pg_stext, Oid nspoid, + const char *stxname); +static bool delete_pg_statistic_ext_data(Oid stxoid, bool inherited); + +typedef struct +{ + bool ndistinct; + bool dependencies; + bool mcv; + bool expressions; +} stakindFlags; + +static void expand_stxkind(HeapTuple tup, stakindFlags * enabled); +static void upsert_pg_statistic_ext_data(const Datum *values, + const bool *nulls, + const bool *replaces); +static bool check_mcvlist_array(ArrayType *arr, int argindex, + int required_ndims, int mcv_length); +static Datum import_expressions(Relation pgsd, int numexprs, + Oid *atttypids, int32 *atttypmods, + Oid *atttypcolls, ArrayType *exprs_arr); +static bool text_to_float4(Datum input, Datum *output); +static bool text_to_int4(Datum input, Datum *output); + =20 /* * Compute requested extended stats, using the rows sampled for the plain @@ -2611,3 +2720,1099 @@ make_build_data(Relation rel, StatExtEntry *stat, = int numrows, HeapTuple *rows, =20 return result; } + +/* + * Fetch a pg_statistic_ext row by name+nspoid. + */ +static HeapTuple +get_pg_statistic_ext(Relation pg_stext, Oid nspoid, const char *stxname) +{ + ScanKeyData key[2]; + SysScanDesc scan; + HeapTuple tup; + Oid stxoid =3D InvalidOid; + + ScanKeyInit(&key[0], + Anum_pg_statistic_ext_stxname, + BTEqualStrategyNumber, + F_NAMEEQ, + CStringGetDatum(stxname)); + ScanKeyInit(&key[1], + Anum_pg_statistic_ext_stxnamespace, + BTEqualStrategyNumber, + F_OIDEQ, + ObjectIdGetDatum(nspoid)); + + /* + * Try to find matching pg_statistic_ext row. + */ + scan =3D systable_beginscan(pg_stext, + StatisticExtNameIndexId, + true, + NULL, + 2, + key); + + /* Unique index, so we get 0 or 1 tuples. */ + tup =3D systable_getnext(scan); + + if (HeapTupleIsValid(tup)) + stxoid =3D ((Form_pg_statistic_ext) GETSTRUCT(tup))->oid; + + systable_endscan(scan); + + if (!OidIsValid(stxoid)) + return NULL; + + return SearchSysCacheCopy1(STATEXTOID, ObjectIdGetDatum(stxoid)); +} + +/* + * Decode the stxkind column so that we know which stats types to expect. + */ +static void +expand_stxkind(HeapTuple tup, stakindFlags * enabled) +{ + Datum datum; + ArrayType *arr; + char *kinds; + + datum =3D SysCacheGetAttrNotNull(STATEXTOID, + tup, + Anum_pg_statistic_ext_stxkind); + arr =3D DatumGetArrayTypeP(datum); + if (ARR_NDIM(arr) !=3D 1 || ARR_HASNULL(arr) || ARR_ELEMTYPE(arr) !=3D CH= AROID) + elog(ERROR, "stxkind is not a 1-D char array"); + + kinds =3D (char *) ARR_DATA_PTR(arr); + + for (int i =3D 0; i < ARR_DIMS(arr)[0]; i++) + if (kinds[i] =3D=3D STATS_EXT_NDISTINCT) + enabled->ndistinct =3D true; + else if (kinds[i] =3D=3D STATS_EXT_DEPENDENCIES) + enabled->dependencies =3D true; + else if (kinds[i] =3D=3D STATS_EXT_MCV) + enabled->mcv =3D true; + else if (kinds[i] =3D=3D STATS_EXT_EXPRESSIONS) + enabled->expressions =3D true; +} + +/* + * Perform the actual storage of a pg_statistic_ext_data tuple. + */ +static void +upsert_pg_statistic_ext_data(const Datum *values, const bool *nulls, + const bool *replaces) +{ + Relation pg_stextdata; + HeapTuple stxdtup; + HeapTuple newtup; + + pg_stextdata =3D table_open(StatisticExtDataRelationId, RowExclusiveLock); + + stxdtup =3D SearchSysCache2(STATEXTDATASTXOID, + values[Anum_pg_statistic_ext_data_stxoid - 1], + values[Anum_pg_statistic_ext_data_stxdinherit - 1]); + + if (HeapTupleIsValid(stxdtup)) + { + newtup =3D heap_modify_tuple(stxdtup, + RelationGetDescr(pg_stextdata), + values, + nulls, + replaces); + CatalogTupleUpdate(pg_stextdata, &newtup->t_self, newtup); + ReleaseSysCache(stxdtup); + } + else + { + newtup =3D heap_form_tuple(RelationGetDescr(pg_stextdata), values, nulls= ); + CatalogTupleInsert(pg_stextdata, newtup); + } + + heap_freetuple(newtup); + + CommandCounterIncrement(); + + table_close(pg_stextdata, RowExclusiveLock); +} + +/* + * Insert or Update Extended Statistics + * + * Major errors, such as the table not existing, the statistics object not + * existing, or a permissions failure are always reported at ERROR. Other + * errors, such as a conversion failure on one statistic kind, are reported + * as WARNINGs, and other statistic kinds may still be updated. + */ +static bool +extended_statistics_update(FunctionCallInfo fcinfo) +{ + Oid nspoid; + char *nspname; + char *stxname; + bool inherited; + Relation pg_stext; + HeapTuple tup =3D NULL; + + stakindFlags enabled; + stakindFlags has; + + Form_pg_statistic_ext stxform; + + Datum values[Natts_pg_statistic_ext_data]; + bool nulls[Natts_pg_statistic_ext_data]; + bool replaces[Natts_pg_statistic_ext_data]; + + bool success =3D true; + + Datum exprdatum; + bool isnull; + List *exprs =3D NIL; + int numattnums =3D 0; + int numexprs =3D 0; + int numattrs =3D 0; + + /* arrays of type info, if we need them */ + Oid *atttypids =3D NULL; + int32 *atttypmods =3D NULL; + Oid *atttypcolls =3D NULL; + Relation rel; + Oid locked_table =3D InvalidOid; + + memset(nulls, false, sizeof(nulls)); + memset(values, 0, sizeof(values)); + memset(replaces, 0, sizeof(replaces)); + memset(&enabled, 0, sizeof(enabled)); + + has.mcv =3D (!PG_ARGISNULL(MOST_COMMON_VALS_ARG) && + !PG_ARGISNULL(MOST_COMMON_VAL_NULLS_ARG) && + !PG_ARGISNULL(MOST_COMMON_FREQS_ARG) && + !PG_ARGISNULL(MOST_COMMON_BASE_FREQS_ARG)); + has.ndistinct =3D !PG_ARGISNULL(NDISTINCT_ARG); + has.dependencies =3D !PG_ARGISNULL(DEPENDENCIES_ARG); + has.expressions =3D !PG_ARGISNULL(EXPRESSIONS_ARG); + + if (RecoveryInProgress()) + { + ereport(WARNING, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("Statistics cannot be modified during recovery.")); + PG_RETURN_BOOL(false); + } + + stats_check_required_arg(fcinfo, extarginfo, STATSCHEMA_ARG); + nspname =3D TextDatumGetCString(PG_GETARG_DATUM(STATSCHEMA_ARG)); + stats_check_required_arg(fcinfo, extarginfo, STATNAME_ARG); + stxname =3D TextDatumGetCString(PG_GETARG_DATUM(STATNAME_ARG)); + stats_check_required_arg(fcinfo, extarginfo, INHERITED_ARG); + inherited =3D PG_GETARG_NAME(INHERITED_ARG); + + nspoid =3D get_namespace_oid(nspname, true); + if (nspoid =3D=3D InvalidOid) + { + ereport(WARNING, + errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Namespace \"%s\" not found.", stxname)); + PG_RETURN_BOOL(false); + } + + pg_stext =3D table_open(StatisticExtRelationId, RowExclusiveLock); + tup =3D get_pg_statistic_ext(pg_stext, nspoid, stxname); + + if (!HeapTupleIsValid(tup)) + { + table_close(pg_stext, RowExclusiveLock); + ereport(WARNING, + errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Extended Statistics Object \"%s\".\"%s\" not found.", + get_namespace_name(nspoid), stxname)); + PG_RETURN_BOOL(false); + } + + stxform =3D (Form_pg_statistic_ext) GETSTRUCT(tup); + expand_stxkind(tup, &enabled); + numattnums =3D stxform->stxkeys.dim1; + + /* decode expression (if any) */ + exprdatum =3D SysCacheGetAttr(STATEXTOID, + tup, + Anum_pg_statistic_ext_stxexprs, + &isnull); + + if (!isnull) + { + char *s; + + s =3D TextDatumGetCString(exprdatum); + exprs =3D (List *) stringToNode(s); + pfree(s); + + /* + * Run the expressions through eval_const_expressions. This is not + * just an optimization, but is necessary, because the planner will be + * comparing them to similarly-processed qual clauses, and may fail to + * detect valid matches without this. We must not use + * canonicalize_qual, however, since these aren't qual expressions. + */ + exprs =3D (List *) eval_const_expressions(NULL, (Node *) exprs); + + /* May as well fix opfuncids too */ + fix_opfuncids((Node *) exprs); + } + numexprs =3D list_length(exprs); + numattrs =3D numattnums + numexprs; + + /* Roundabout way of getting a RangeVar on the underlying table */ + rel =3D relation_open(stxform->stxrelid, AccessShareLock); + + /* no need to fetch reloid, we already have it */ + RangeVarGetRelidExtended(makeRangeVar(nspname, + RelationGetRelationName(rel), -1), + ShareUpdateExclusiveLock, 0, + RangeVarCallbackForStats, &locked_table); + + relation_close(rel, AccessShareLock); + + if (has.mcv) + { + if (!enabled.mcv) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV parameters \"%s\", \"%s\", \"%s\", and \"%s\" were all " + "specified for extended statistics object that does not expect MC= V ", + extarginfo[MOST_COMMON_VALS_ARG].argname, + extarginfo[MOST_COMMON_VAL_NULLS_ARG].argname, + extarginfo[MOST_COMMON_FREQS_ARG].argname, + extarginfo[MOST_COMMON_BASE_FREQS_ARG].argname)); + has.mcv =3D false; + success =3D false; + } + } + else + { + /* The MCV args must all be NULL. */ + if (!PG_ARGISNULL(MOST_COMMON_VALS_ARG) || + !PG_ARGISNULL(MOST_COMMON_VAL_NULLS_ARG) || + !PG_ARGISNULL(MOST_COMMON_FREQS_ARG) || + !PG_ARGISNULL(MOST_COMMON_BASE_FREQS_ARG)) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV parameters \"%s\", \"%s\", \"%s\", and \"%s\" must be all= specified if any are specified", + extarginfo[MOST_COMMON_VALS_ARG].argname, + extarginfo[MOST_COMMON_VAL_NULLS_ARG].argname, + extarginfo[MOST_COMMON_FREQS_ARG].argname, + extarginfo[MOST_COMMON_BASE_FREQS_ARG].argname)); + success =3D false; + } + } + + if (has.ndistinct && !enabled.ndistinct) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics object= " + "that does not expect \"%s\"", + extarginfo[NDISTINCT_ARG].argname, + extarginfo[NDISTINCT_ARG].argname)); + has.ndistinct =3D false; + success =3D false; + } + + if (has.dependencies && !enabled.dependencies) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics object= " + "that does not expect \"%s\"", + extarginfo[DEPENDENCIES_ARG].argname, + extarginfo[DEPENDENCIES_ARG].argname)); + has.dependencies =3D false; + success =3D false; + } + + if (has.expressions && !enabled.expressions) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics object= " + "that does not expect \"%s\"", + extarginfo[DEPENDENCIES_ARG].argname, + extarginfo[DEPENDENCIES_ARG].argname)); + has.expressions =3D false; + success =3D false; + } + + /* + * Either of these statsistic types requires that we supply + * semi-filled-out VacAttrStatP array. + * + * + * It is not possible to use the existing lookup_var_attr_stats() and + * examine_attribute() because these functions will skip attributes for + * which attstattarget is 0, and we may have stats to import for those + * attributes. + */ + if (has.mcv || has.expressions) + { + atttypids =3D palloc0_array(Oid, numattrs); + atttypmods =3D palloc0_array(int32, numattrs); + atttypcolls =3D palloc0_array(Oid, numattrs); + + for (int i =3D 0; i < numattnums; i++) + { + AttrNumber attnum =3D stxform->stxkeys.values[i]; + + Oid lt_opr; + Oid eq_opr; + char typetype; + + /* + * fetch attribute entries the same as are done for attribute + * stats + */ + statatt_get_type(stxform->stxrelid, + attnum, + &atttypids[i], + &atttypmods[i], + &typetype, + &atttypcolls[i], + <_opr, + &eq_opr); + } + + for (int i =3D numattnums; i < numattrs; i++) + { + Node *expr =3D list_nth(exprs, i - numattnums); + + atttypids[i] =3D exprType(expr); + atttypmods[i] =3D exprTypmod(expr); + atttypcolls[i] =3D exprCollation(expr); + + /* + * Duplicate logic from get_attr_stat_type + */ + + /* + * If it's a multirange, step down to the range type, as is done + * by multirange_typanalyze(). + */ + if (type_is_multirange(atttypids[i])) + atttypids[i] =3D get_multirange_range(atttypids[i]); + + /* + * Special case: collation for tsvector is DEFAULT_COLLATION_OID. + * See compute_tsvector_stats(). + */ + if (atttypids[i] =3D=3D TSVECTOROID) + atttypcolls[i] =3D DEFAULT_COLLATION_OID; + + } + } + + /* Primary Key: cannot be NULL or replaced. */ + values[Anum_pg_statistic_ext_data_stxoid - 1] =3D ObjectIdGetDatum(stxfor= m->oid); + values[Anum_pg_statistic_ext_data_stxdinherit - 1] =3D BoolGetDatum(inher= ited); + + if (has.ndistinct) + { + Datum ndistinct_datum =3D PG_GETARG_DATUM(NDISTINCT_ARG); + bytea *data =3D DatumGetByteaPP(ndistinct_datum); + MVNDistinct *ndistinct =3D statext_ndistinct_deserialize(data); + + if (pg_ndistinct_validate_items(ndistinct, &stxform->stxkeys, numexprs, = WARNING)) + { + values[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D ndistinct_datu= m; + replaces[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + } + else + { + nulls[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + success =3D false; + } + + free_pg_ndistinct(ndistinct); + } + else + nulls[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + + if (has.dependencies) + { + Datum dependencies_datum =3D PG_GETARG_DATUM(DEPENDENCIES_ARG); + bytea *data =3D DatumGetByteaPP(dependencies_datum); + MVDependencies *dependencies =3D statext_dependencies_deserialize(data); + + if (pg_dependencies_validate_deps(dependencies, &stxform->stxkeys, numex= prs, WARNING)) + { + values[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D dependencie= s_datum; + replaces[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + } + else + { + nulls[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + success =3D false; + } + + free_pg_dependencies(dependencies); + } + else + nulls[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + + if (has.mcv) + { + Datum datum; + ArrayType *mcv_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_VALS_ARG); + ArrayType *nulls_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_VAL_NULLS_AR= G); + ArrayType *freqs_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_FREQS_ARG); + ArrayType *base_freqs_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_BASE_FR= EQS_ARG); + int nitems; + Datum *mcv_elems; + bool *mcv_nulls; + int check_nummcv; + + /* + * The mcv_arr is an array of arrays of text, and we use it as the + * reference array for checking the lengths of the other 3 arrays. + */ + if (ARR_NDIM(mcv_arr) !=3D 2) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" must be a text array of 2 dimensions.", + extarginfo[MOST_COMMON_VALS_ARG].argname)); + return (Datum) 0; + } + + nitems =3D ARR_DIMS(mcv_arr)[0]; + + /* Fixed length arrays that cannot contain NULLs. */ + if (!check_mcvlist_array(nulls_arr, MOST_COMMON_VAL_NULLS_ARG, + 2, nitems) || + !check_mcvlist_array(freqs_arr, MOST_COMMON_FREQS_ARG, + 1, nitems) || + !check_mcvlist_array(base_freqs_arr, MOST_COMMON_BASE_FREQS_ARG, + 1, nitems)) + return (Datum) 0; + + + deconstruct_array_builtin(mcv_arr, TEXTOID, &mcv_elems, + &mcv_nulls, &check_nummcv); + + Assert(check_nummcv =3D=3D (nitems * numattrs)); + + datum =3D import_mcvlist(tup, WARNING, numattrs, + atttypids, atttypmods, atttypcolls, + nitems, mcv_elems, mcv_nulls, + (bool *) ARR_DATA_PTR(nulls_arr), + (float8 *) ARR_DATA_PTR(freqs_arr), + (float8 *) ARR_DATA_PTR(base_freqs_arr)); + + values[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D datum; + replaces[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D true; + } + else + nulls[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D true; + + if (has.expressions) + { + Datum datum; + Relation pgsd; + + pgsd =3D table_open(StatisticRelationId, RowExclusiveLock); + + /* + * Generate the expressions array. + * + * The attytypids, attytypmods, and atttypcols arrays have all the regul= ar + * attributes listed first, so we can pass those arrays with a start poi= nt + * after the last regular attribute, and there should be numexprs elemen= ts + * remaining. + */ + datum =3D import_expressions(pgsd, numexprs, + &atttypids[numattnums], &atttypmods[numattnums], + &atttypcolls[numattnums], + PG_GETARG_ARRAYTYPE_P(EXPRESSIONS_ARG)); + + table_close(pgsd, RowExclusiveLock); + + values[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D datum; + replaces[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D true; + } + else + nulls[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D true; + + upsert_pg_statistic_ext_data(values, nulls, replaces); + + heap_freetuple(tup); + table_close(pg_stext, RowExclusiveLock); + + if (atttypids !=3D NULL) + pfree(atttypids); + if (atttypmods !=3D NULL) + pfree(atttypmods); + if (atttypcolls !=3D NULL) + pfree(atttypcolls); + return success; +} + +/* + * Consistency checks to ensure that other mcvlist arrays are in alignment + * with the mcv array. + */ +static bool +check_mcvlist_array(ArrayType *arr, int argindex, int required_ndims, + int mcv_length) +{ + if (ARR_NDIM(arr) !=3D required_ndims) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must be an array of %d dimensions.", + extarginfo[argindex].argname, required_ndims)); + return false; + } + + if (array_contains_nulls(arr)) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Array \"%s\" cannot contain NULLs.", + extarginfo[argindex].argname)); + return false; + } + + if (ARR_DIMS(arr)[0] !=3D mcv_length) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" must have the same number of elements as \"%= s\"", + extarginfo[argindex].argname, + extarginfo[MOST_COMMON_VALS_ARG].argname)); + return false; + } + + return true; +} + +/* + * Warn of type mismatch. Common pattern. + */ +static Datum +warn_type_mismatch(Datum d, const char *argname) +{ + char *s =3D TextDatumGetCString(d); + + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input type= =2E", + argname, s)); + return (Datum) 0; +} + +/* + * Create the stxdexprs datum using the user input in an array of array of + * text, referenced against the datatypes for the expressions. + * + * This datum is needed to fill out a complete pg_statistic_ext_data tuple. + * + * The input arrays should each have numexprs elements in them and they sh= ould + * be the in the order that the expressions appear in the statistics objec= t. + */ +static Datum +import_expressions(Relation pgsd, int numexprs, + Oid *atttypids, int32 *atttypmods, + Oid *atttypcolls, ArrayType *exprs_arr) +{ + Datum *exprs_elems; + bool *exprs_nulls; + int check_numexprs; + int offset =3D 0; + + FmgrInfo array_in_fn; + + Oid pgstypoid =3D get_rel_type_id(StatisticRelationId); + + ArrayBuildState *astate =3D NULL; + + /* + * Verify that the exprs_array is something that matches the expectations + * set by stxdexprs generally and the specific statistics object definiti= on. + */ + if (ARR_NDIM(exprs_arr) !=3D 2) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must be a text array of 2 dimensions.", + extarginfo[EXPRESSIONS_ARG].argname)); + return (Datum) 0; + } + + if (ARR_DIMS(exprs_arr)[0] !=3D numexprs) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must have an outer dimension of %d elements.", + extarginfo[EXPRESSIONS_ARG].argname, numexprs)); + return (Datum) 0; + } + if (ARR_DIMS(exprs_arr)[1] !=3D NUM_ATTRIBUTE_STATS_ELEMS) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must have an inner dimension of %d elements.", + extarginfo[EXPRESSIONS_ARG].argname, + NUM_ATTRIBUTE_STATS_ELEMS)); + return (Datum) 0; + } + + fmgr_info(F_ARRAY_IN, &array_in_fn); + + deconstruct_array_builtin(exprs_arr, TEXTOID, &exprs_elems, + &exprs_nulls, &check_numexprs); + + /* + * Iterate over each expected expression. + * + * The values/nulls/replaces arrays are deconstructed into a 1-D arrays, = so + * we have to advance an offset by NUM_ATTRIBUTE_STATS_ELEMS to get to the + * next row of the 2-D array. + */ + for (int i =3D 0; i < numexprs; i++) + { + Oid typid =3D atttypids[i]; + int32 typmod =3D atttypmods[i]; + Oid stacoll =3D atttypcolls[i]; + TypeCacheEntry *typcache; + + Oid elemtypid =3D InvalidOid; + Oid elem_eq_opr =3D InvalidOid; + + bool ok; + + Datum values[Natts_pg_statistic]; + bool nulls[Natts_pg_statistic]; + bool replaces[Natts_pg_statistic]; + + HeapTuple pgstup; + Datum pgstdat; + + /* Advance the indexes to the next offset. */ + const int null_frac_idx =3D offset + NULL_FRAC_ELEM; + const int avg_width_idx =3D offset + AVG_WIDTH_ELEM; + const int n_distinct_idx =3D offset + N_DISTINCT_ELEM; + const int most_common_vals_idx =3D offset + MOST_COMMON_VALS_ELEM; + const int most_common_freqs_idx =3D offset + MOST_COMMON_FREQS_ELEM; + const int histogram_bounds_idx =3D offset + HISTOGRAM_BOUNDS_ELEM; + const int correlation_idx =3D offset + CORRELATION_ELEM; + const int most_common_elems_idx =3D offset + MOST_COMMON_ELEMS_ELEM; + const int most_common_elems_freqs_idx =3D offset + MOST_COMMON_ELEM_FREQ= S_ELEM; + const int elem_count_histogram_idx =3D offset + ELEM_COUNT_HISTOGRAM_ELE= M; + + /* This finds the right operators even if atttypid is a domain */ + typcache =3D lookup_type_cache(typid, TYPECACHE_LT_OPR | TYPECACHE_EQ_OP= R); + + statatt_init_empty_tuple(InvalidOid, InvalidAttrNumber, false, + values, nulls, replaces); + + /* + * Check each of the fixed attributes to see if they have values set. If= not + * set, then just let them stay with the default values set in + * statatt_init_empty_tuple(). + */ + if (!exprs_nulls[null_frac_idx]) + { + ok =3D text_to_float4(exprs_elems[null_frac_idx], + &values[Anum_pg_statistic_stanullfrac - 1]); + + if (!ok) + return warn_type_mismatch(exprs_elems[null_frac_idx], + extexprarginfo[NULL_FRAC_ELEM].argname); + } + + if (!exprs_nulls[avg_width_idx]) + { + ok =3D text_to_int4(exprs_elems[avg_width_idx], + &values[Anum_pg_statistic_stawidth - 1]); + + if (!ok) + return warn_type_mismatch(exprs_elems[avg_width_idx], + extexprarginfo[AVG_WIDTH_ELEM].argname); + } + + if (!exprs_nulls[n_distinct_idx]) + { + ok =3D text_to_float4(exprs_elems[n_distinct_idx], + &values[Anum_pg_statistic_stadistinct - 1]); + + if (!ok) + return warn_type_mismatch(exprs_elems[n_distinct_idx], + extexprarginfo[N_DISTINCT_ELEM].argname); + } + + /* + * The STAKIND statistics are the same as the ones found in attribute + * stats. However, these are all derived from text columns, whereas + * the ones derived for attribute stats are a mix of datatypes. This + * limits the opportunities for code sharing between the two. + * + * Some statistic kinds have both a stanumbers and a stavalues component= s. + * In those cases, both values must either be NOT NULL or both NULL, and + * if they aren't then we need to reject that stakind completely. Curren= tly + * we go a step further and reject the expression array completely. + * + * Once it is established that the pairs are in NULL/NOT-NULL alignment, + * we can test either expr_nulls[] value to see if the stakind has value= (s) + * that we can set or not. + */ + + /* STATISTIC_KIND_MCV */ + if (exprs_nulls[most_common_vals_idx] !=3D + exprs_nulls[most_common_freqs_idx]) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s and %s must both be NOT NULL or both NULL.", + extexprarginfo[MOST_COMMON_VALS_ELEM].argname, + extexprarginfo[MOST_COMMON_FREQS_ELEM].argname)); + return (Datum) 0; + } + + if (!exprs_nulls[most_common_vals_idx]) + { + Datum stavalues; + Datum stanumbers; + + stavalues =3D statatt_build_stavalues(extexprarginfo[MOST_COMMON_VALS_E= LEM].argname, + &array_in_fn, exprs_elems[most_common_vals_idx], + typid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + stanumbers =3D statatt_build_stavalues(extexprarginfo[MOST_COMMON_FREQS= _ELEM].argname, + &array_in_fn, exprs_elems[most_common_freqs_idx], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCV, + typcache->eq_opr, stacoll, + stanumbers, false, stavalues, false); + } + + /* STATISTIC_KIND_HISTOGRAM */ + if (!exprs_nulls[histogram_bounds_idx]) + { + Datum stavalues; + + stavalues =3D statatt_build_stavalues(extexprarginfo[HISTOGRAM_BOUNDS_E= LEM].argname, + &array_in_fn, exprs_elems[histogram_bounds_idx], + typid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_HISTOGRAM, + typcache->lt_opr, stacoll, + 0, true, stavalues, false); + } + + /* STATISTIC_KIND_CORRELATION */ + if (!exprs_nulls[correlation_idx]) + { + Datum corr[] =3D {(Datum) 0}; + ArrayType *arry; + Datum stanumbers; + + ok =3D text_to_float4(exprs_elems[correlation_idx], &corr[0]); + + if (!ok) + { + char *s =3D TextDatumGetCString(exprs_elems[correlation_idx]); + + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input t= ype.", + extexprarginfo[CORRELATION_ELEM].argname, s)); + return (Datum) 0; + } + + arry =3D construct_array_builtin(corr, 1, FLOAT4OID); + + stanumbers =3D PointerGetDatum(arry); + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_CORRELATION, + typcache->lt_opr, stacoll, + stanumbers, false, 0, true); + } + + /* STATISTIC_KIND_MCELEM */ + if (exprs_nulls[most_common_elems_idx] !=3D + exprs_nulls[most_common_elems_freqs_idx]) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s and %s must both be NOT NULL or both NULL.", + extexprarginfo[MOST_COMMON_ELEMS_ELEM].argname, + extexprarginfo[MOST_COMMON_ELEM_FREQS_ELEM].argname)); + return (Datum) 0; + } + + /* + * We only need to fetch element type and eq operator if we have a + * stat of type MCELEM or DECHIST, otherwise the values are unnecessary + * and not meaningful. + */ + if (!exprs_nulls[most_common_elems_idx] || + !exprs_nulls[elem_count_histogram_idx]) + { + if (!statatt_get_elem_type(typid, typcache->typtype, + &elemtypid, &elem_eq_opr)) + { + ereport(WARNING, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("unable to determine element type of expression")); + return (Datum) 0; + } + } + + if (!exprs_nulls[most_common_elems_idx]) + { + Datum stavalues; + Datum stanumbers; + + stavalues =3D statatt_build_stavalues(extexprarginfo[MOST_COMMON_ELEMS_= ELEM].argname, + &array_in_fn, + exprs_elems[most_common_elems_idx], + elemtypid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + stanumbers =3D statatt_build_stavalues(extexprarginfo[MOST_COMMON_ELEM_= FREQS_ELEM].argname, + &array_in_fn, + exprs_elems[most_common_elems_freqs_idx], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCELEM, + elem_eq_opr, stacoll, + stanumbers, false, stavalues, false); + } + + if (!exprs_nulls[elem_count_histogram_idx]) + { + Datum stanumbers; + + stanumbers =3D statatt_build_stavalues(extexprarginfo[ELEM_COUNT_HISTOG= RAM_ELEM].argname, + &array_in_fn, + exprs_elems[elem_count_histogram_idx], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, STATISTIC_KIND_DECHIST, + elem_eq_opr, stacoll, + stanumbers, false, 0, true); + } + + /* + * Currently there are no extended stats exports of the statistic + * kinds STATISTIC_KIND_BOUNDS_HISTOGRAM or + * STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM so these cannot be imported. + * These may be added in the future. + */ + + pgstup =3D heap_form_tuple(RelationGetDescr(pgsd), values, nulls); + pgstdat =3D heap_copy_tuple_as_datum(pgstup, RelationGetDescr(pgsd)); + astate =3D accumArrayResult(astate, pgstdat, false, pgstypoid, + CurrentMemoryContext); + + offset +=3D NUM_ATTRIBUTE_STATS_ELEMS; + } + + pfree(exprs_elems); + pfree(exprs_nulls); + + return makeArrayResult(astate, CurrentMemoryContext); +} + +/* + * Safe conversion of text to float4. + * + * There is no need for the specific error message. + */ +static bool +text_to_float4(Datum input, Datum *output) +{ + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + char *s; + bool ok; + + s =3D TextDatumGetCString(input); + ok =3D DirectInputFunctionCallSafe(float4in, s, InvalidOid, -1, + (Node *) &escontext, output); + + pfree(s); + return ok; +} + + +/* + * Safe conversion of text to int4. + * + * There is no need for the specific error message. + */ +static bool +text_to_int4(Datum input, Datum *output) +{ + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + char *s; + bool ok; + + s =3D TextDatumGetCString(input); + ok =3D DirectInputFunctionCallSafe(int4in, s, InvalidOid, -1, + (Node *) &escontext, output); + + pfree(s); + return ok; +} + +/* + * Remove an existing pg_statistic_ext_data row for a given pg_statistic_e= xt + * row + inherited pair. + */ +static bool +delete_pg_statistic_ext_data(Oid stxoid, bool inherited) +{ + Relation sed =3D table_open(StatisticExtDataRelationId, RowExclusiveLock); + HeapTuple oldtup; + bool result =3D false; + + /* Is there already a pg_statistic tuple for this attribute? */ + oldtup =3D SearchSysCache2(STATEXTDATASTXOID, + ObjectIdGetDatum(stxoid), + BoolGetDatum(inherited)); + + if (HeapTupleIsValid(oldtup)) + { + CatalogTupleDelete(sed, &oldtup->t_self); + ReleaseSysCache(oldtup); + result =3D true; + } + + table_close(sed, RowExclusiveLock); + + CommandCounterIncrement(); + + return result; +} + +/* + * Restore (insert or replace) statistics for the given statistics object. + */ +Datum +pg_restore_extended_stats(PG_FUNCTION_ARGS) +{ + LOCAL_FCINFO(positional_fcinfo, NUM_EXTENDED_STATS_ARGS); + bool result =3D true; + + InitFunctionCallInfoData(*positional_fcinfo, NULL, NUM_EXTENDED_STATS_ARG= S, + InvalidOid, NULL, NULL); + + if (!stats_fill_fcinfo_from_arg_pairs(fcinfo, positional_fcinfo, extargin= fo)) + result =3D false; + + if (!extended_statistics_update(positional_fcinfo)) + result =3D false; + + PG_RETURN_BOOL(result); +} + +/* + * Delete statistics for the given statistics object. + */ +Datum +pg_clear_extended_stats(PG_FUNCTION_ARGS) +{ + char *nspname; + Oid nspoid; + char *stxname; + bool inherited; + Relation pg_stext; + HeapTuple tup; + Relation rel; + Oid locked_table =3D InvalidOid; + + Form_pg_statistic_ext stxform; + + stats_check_required_arg(fcinfo, extarginfo, STATSCHEMA_ARG); + nspname =3D TextDatumGetCString(PG_GETARG_DATUM(STATSCHEMA_ARG)); + stats_check_required_arg(fcinfo, extarginfo, STATNAME_ARG); + stxname =3D TextDatumGetCString(PG_GETARG_DATUM(STATNAME_ARG)); + stats_check_required_arg(fcinfo, extarginfo, INHERITED_ARG); + inherited =3D PG_GETARG_NAME(INHERITED_ARG); + + if (RecoveryInProgress()) + { + ereport(WARNING, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("Statistics cannot be modified during recovery.")); + PG_RETURN_VOID(); + } + + nspoid =3D get_namespace_oid(nspname, true); + if (nspoid =3D=3D InvalidOid) + { + ereport(WARNING, + errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Namespace \"%s\" not found.", stxname)); + PG_RETURN_VOID(); + } + + pg_stext =3D table_open(StatisticExtRelationId, RowExclusiveLock); + tup =3D get_pg_statistic_ext(pg_stext, nspoid, stxname); + + if (!HeapTupleIsValid(tup)) + { + table_close(pg_stext, RowExclusiveLock); + ereport(ERROR, + errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Extended Statistics Object \"%s\".\"%s\" not found.", + nspname, stxname)); + PG_RETURN_VOID(); + } + + stxform =3D (Form_pg_statistic_ext) GETSTRUCT(tup); + + /* Roundabout way of getting a RangeVar on the underlying table */ + rel =3D relation_open(stxform->stxrelid, AccessShareLock); + + /* no need to fetch reloid, we already have it */ + RangeVarGetRelidExtended(makeRangeVar(nspname, + RelationGetRelationName(rel), -1), + ShareUpdateExclusiveLock, 0, + RangeVarCallbackForStats, &locked_table); + + relation_close(rel, AccessShareLock); + + delete_pg_statistic_ext_data(stxform->oid, inherited); + heap_freetuple(tup); + table_close(pg_stext, RowExclusiveLock); + + PG_RETURN_VOID(); +} diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c index ec650ba029f5..d0113dceda05 100644 --- a/src/backend/statistics/mcv.c +++ b/src/backend/statistics/mcv.c @@ -2173,3 +2173,147 @@ mcv_clause_selectivity_or(PlannerInfo *root, Statis= ticExtInfo *stat, =20 return s; } + +/* + * The MCV is an array of records, but this is expected as 4 separate arr= ays. + * It is not possible to have a generic input function for pg_mcv_list + * because the most_common_values is a composite type with element types + * defined by the specific statistics object. + */ +Datum +import_mcvlist(HeapTuple tup, int elevel, int numattrs, Oid *atttypids, + int32 *atttypmods, Oid *atttypcolls, int nitems, + Datum *mcv_elems, bool *mcv_nulls, + bool *mcv_elem_nulls, float8 *freqs, float8 *base_freqs) +{ + MCVList *mcvlist; + bytea *bytes; + + HeapTuple *vatuples; + VacAttrStats **vastats; + + /* + * Allocate the MCV list structure, set the global parameters. + */ + mcvlist =3D (MCVList *) palloc0(offsetof(MCVList, items) + + (sizeof(MCVItem) * nitems)); + + mcvlist->magic =3D STATS_MCV_MAGIC; + mcvlist->type =3D STATS_MCV_TYPE_BASIC; + mcvlist->ndimensions =3D numattrs; + mcvlist->nitems =3D nitems; + + /* Set the values for the 1-D arrays and allocate space for the 2-D array= s */ + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + item->frequency =3D freqs[i]; + item->base_frequency =3D base_freqs[i]; + item->values =3D (Datum *) palloc0_array(Datum, numattrs); + item->isnull =3D (bool *) palloc0_array(bool, numattrs); + } + + /* Walk through each dimension */ + for (int j =3D 0; j < numattrs; j++) + { + FmgrInfo finfo; + Oid ioparam; + Oid infunc; + int index =3D j; + + getTypeInputInfo(atttypids[j], &infunc, &ioparam); + fmgr_info(infunc, &finfo); + + /* store info about data type OIDs */ + mcvlist->types[j] =3D atttypids[j]; + + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + /* These should be in agreement, but just to be safe check both */ + if (mcv_elem_nulls[index] || mcv_nulls[index]) + { + item->values[j] =3D (Datum) 0; + item->isnull[j] =3D true; + } + else + { + char *s =3D TextDatumGetCString(mcv_elems[index]); + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + if (!InputFunctionCallSafe(&finfo, s, ioparam, atttypmods[j], + (Node *) &escontext, &item->values[j])) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV elemement \"%s\" does not match expected input type.",= s))); + return (Datum) 0; + } + + pfree(s); + } + + index +=3D numattrs; + } + } + + /* + * The function statext_mcv_serialize() requires an array of pointers to + * VacAttrStats records, but only a few fields within those records have + * to be filled out. + */ + vastats =3D (VacAttrStats **) palloc0_array(VacAttrStats *, numattrs); + vatuples =3D (HeapTuple *) palloc0_array(HeapTuple, numattrs); + + for (int i =3D 0; i < numattrs; i++) + { + Oid typid =3D atttypids[i]; + HeapTuple typtuple; + + typtuple =3D SearchSysCacheCopy1(TYPEOID, ObjectIdGetDatum(typid)); + + if (!HeapTupleIsValid(typtuple)) + elog(ERROR, "cache lookup failed for type %u", typid); + + vatuples[i] =3D typtuple; + + vastats[i] =3D palloc0_object(VacAttrStats); + + vastats[i]->attrtype =3D (Form_pg_type) GETSTRUCT(typtuple); + vastats[i]->attrtypid =3D typid; + vastats[i]->attrcollid =3D atttypcolls[i]; + } + + bytes =3D statext_mcv_serialize(mcvlist, vastats); + + for (int i =3D 0; i < numattrs; i++) + { + pfree(vatuples[i]); + pfree(vastats[i]); + } + pfree((void *) vatuples); + pfree((void *) vastats); + + if (bytes =3D=3D NULL) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Unable to import mcv list"))); + return (Datum) 0; + } + + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + pfree(item->values); + pfree(item->isnull); + } + pfree(mcvlist); + pfree(mcv_elems); + pfree(mcv_nulls); + + return PointerGetDatum(bytes); +} diff --git a/src/backend/statistics/mvdistinct.c b/src/backend/statistics/m= vdistinct.c index 58046d2bd625..d189602bb35b 100644 --- a/src/backend/statistics/mvdistinct.c +++ b/src/backend/statistics/mvdistinct.c @@ -599,6 +599,68 @@ generate_combinations_recurse(CombinationGenerator *st= ate, } } =20 +/* + * Free allocations of an MVNDistinct + */ +void +free_pg_ndistinct(MVNDistinct *ndistinct) +{ + for (int i =3D 0; i < ndistinct->nitems; i++) + pfree(ndistinct->items[i].attributes); + + pfree(ndistinct); +} + +/* + * Validate an MVNDistinct against the extended statistics object definiti= on. + * + * Every MVNDistinctItem must be checked to ensure that the attnums in the + * attributes list correspond to attnums/expressions defined by the + * extended statistics object. + * + * Positive attnums are attributes which must be found in the stxkeys, + * while negative attnums correspond to an expr number, so the attnum + * can't be below (0 - numexprs). + */ +bool +pg_ndistinct_validate_items(MVNDistinct *ndistinct, int2vector *stxkeys, i= nt numexprs, int elevel) +{ + int attnum_expr_lowbound =3D 0 - numexprs; + + for (int i =3D 0; i < ndistinct->nitems; i++) + { + MVNDistinctItem item =3D ndistinct->items[i]; + + for (int j =3D 0; j < item.nattributes; j++) + { + AttrNumber attnum =3D item.attributes[j]; + bool ok =3D false; + + if (attnum > 0) + { + for (int k =3D 0; k < stxkeys->dim1; k++) + if (attnum =3D=3D stxkeys->values[k]) + { + ok =3D true; + break; + } + } + else if ((attnum < 0) && (attnum >=3D attnum_expr_lowbound)) + ok =3D true; + + if (!ok) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("pg_ndistinct: invalid attnum for this statistics object: %d= ", attnum))); + return false; + } + } + } + return true; +} + + /* * generate_combinations * generate all k-combinations of N elements diff --git a/src/test/regress/expected/stats_import.out b/src/test/regress/= expected/stats_import.out index 98ce7dc28410..4d7dcfca093c 100644 --- a/src/test/regress/expected/stats_import.out +++ b/src/test/regress/expected/stats_import.out @@ -1084,11 +1084,15 @@ SELECT 3, 'tre', (3, 3.3, 'TRE', '2003-03-03', NULL= )::stats_import.complex_type, UNION ALL SELECT 4, 'four', NULL, int4range(0,100), NULL; CREATE INDEX is_odd ON stats_import.test(((comp).a % 2 =3D 1)); +CREATE STATISTICS stats_import.test_stat ON name, comp, lower(arange), arr= ay_length(tags,1) +FROM stats_import.test; -- Generate statistics on table with data ANALYZE stats_import.test; CREATE TABLE stats_import.test_clone ( LIKE stats_import.test ) WITH (autovacuum_enabled =3D false); CREATE INDEX is_odd_clone ON stats_import.test_clone(((comp).a % 2 =3D 1)); +CREATE STATISTICS stats_import.test_stat_clone ON name, comp, lower(arange= ), array_length(tags,1) +FROM stats_import.test_clone; -- -- Copy stats from test to test_clone, and is_odd to is_odd_clone -- @@ -1342,6 +1346,580 @@ AND attname =3D 'i'; (1 row) =20 DROP TABLE stats_temp; +-- set n_distinct using at attnum (1) that is not in the statistics object +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2], "ndistinct" : 4}, + {"attributes" : [1,2,3,-1,-2], "ndistinct" : 4}]':= :pg_ndistinct + ); +WARNING: pg_ndistinct: invalid attnum for this statistics object: 1 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set n_distinct using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [0,2,3,-1,-2], "ndistinct" : 4}]':= :pg_ndistinct + ); +ERROR: malformed pg_ndistinct: "[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [0,2,3,-1,-2], "ndistinct" : 4}]" +LINE 6: 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" ... + ^ +DETAIL: Invalid "attributes" element has been found: 0. +-- set n_distinct using at attnum that is outside the expression bounds(be= low -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [3,-1,-4], "ndistinct" : 4}, + {"attributes" : [3,-2,-4], "ndistinct" : 4}, + {"attributes" : [-1,-2,-4], "ndistinct" : 4}, + {"attributes" : [3,-1,-2,-4], "ndistinct" : 4}]'::= pg_ndistinct + ); +WARNING: pg_ndistinct: invalid attnum for this statistics object: -4 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2], "ndistinct" : 4}]'::p= g_ndistinct + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + e.dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+------------------------------------------------ +n_distinct | [{"attributes": [2, 3], "ndistinct": 4}, + + | {"attributes": [2, -1], "ndistinct": 4}, + + | {"attributes": [3, -1], "ndistinct": 4}, + + | {"attributes": [3, -2], "ndistinct": 4}, + + | {"attributes": [-1, -2], "ndistinct": 3}, + + | {"attributes": [2, 3, -1], "ndistinct": 4}, + + | {"attributes": [2, 3, -2], "ndistinct": 4}, + + | {"attributes": [2, -1, -2], "ndistinct": 4}, + + | {"attributes": [2, 3, -1, -2], "ndistinct": 4}] +dependencies |=20 +most_common_vals |=20 +most_common_val_nulls |=20 +most_common_freqs |=20 +most_common_base_freqs |=20 + +-- set dependencies using at attnum (1) that is not in the statistics obje= ct +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 1, "degree": 1= =2E000000}]'::pg_dependencies + ); +WARNING: pg_dependencies: invalid attnum for this statistics object: 1 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set dependencies using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [0], "dependency": -1, "degree": = 1.000000}]'::pg_dependencies + ); +ERROR: malformed pg_dependencies: "[{"attributes": [0], "dependency": -1,= "degree": 1.000000}]" +LINE 6: 'dependencies', '[{"attributes": [0], "dependency": ... + ^ +DETAIL: Invalid "attributes" element has been found: 0. +-- set dependencies using at attnum that is outside the expression bounds(= below -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": -3, "degree": = 1.000000}]'::pg_dependencies + ); +WARNING: pg_dependencies: invalid attnum for this statistics object: -3 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, + {"attributes": [2], "dependency": -1, "degree": = 1.000000}, + {"attributes": [2], "dependency": -2, "degree": = 1.000000}, + {"attributes": [3], "dependency": 2, "degree": 1= =2E000000}, + {"attributes": [3], "dependency": -1, "degree": = 1.000000}, + {"attributes": [3], "dependency": -2, "degree": = 1.000000}, + {"attributes": [-1], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-1], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-1], "dependency": -2, "degree":= 1.000000}, + {"attributes": [-2], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-2], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-2], "dependency": -1, "degree":= 1.000000}, + {"attributes": [2,3], "dependency": -1, "degree"= : 1.000000}, + {"attributes": [2,3], "dependency": -2, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [2,-2], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [3,-1], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [3,-2], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [-1,-2], "dependency": 2, "degree= ": 0.500000}, + {"attributes": [-1,-2], "dependency": 3, "degree= ": 0.500000}, + {"attributes": [2,3,-2], "dependency": -1, "degr= ee": 1.000000}, + {"attributes": [2,-1,-2], "dependency": 3, "degr= ee": 1.000000}, + {"attributes": [3,-1,-2], "dependency": 2, "degr= ee": 1.000000}]'::pg_dependencies + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + replace(e.dependencies, '}, ', E'},\n') AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+--------------------------------------------------= ---------------- +n_distinct | [{"attributes": [2, 3], "ndistinct": 4}, = + + | {"attributes": [2, -1], "ndistinct": 4}, = + + | {"attributes": [3, -1], "ndistinct": 4}, = + + | {"attributes": [3, -2], "ndistinct": 4}, = + + | {"attributes": [-1, -2], "ndistinct": 3}, = + + | {"attributes": [2, 3, -1], "ndistinct": 4}, = + + | {"attributes": [2, 3, -2], "ndistinct": 4}, = + + | {"attributes": [2, -1, -2], "ndistinct": 4}, = + + | {"attributes": [2, 3, -1, -2], "ndistinct": 4}] +dependencies | [{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, + + | {"attributes": [2], "dependency": -1, "degree": 1= =2E000000}, + + | {"attributes": [2], "dependency": -2, "degree": 1= =2E000000}, + + | {"attributes": [3], "dependency": 2, "degree": 1.= 000000}, + + | {"attributes": [3], "dependency": -1, "degree": 1= =2E000000}, + + | {"attributes": [3], "dependency": -2, "degree": 1= =2E000000}, + + | {"attributes": [-1], "dependency": 2, "degree": 0= =2E500000}, + + | {"attributes": [-1], "dependency": 3, "degree": 0= =2E500000}, + + | {"attributes": [-1], "dependency": -2, "degree": = 1.000000}, + + | {"attributes": [-2], "dependency": 2, "degree": 0= =2E500000}, + + | {"attributes": [-2], "dependency": 3, "degree": 0= =2E500000}, + + | {"attributes": [-2], "dependency": -1, "degree": = 1.000000}, + + | {"attributes": [2, 3], "dependency": -1, "degree"= : 1.000000}, + + | {"attributes": [2, 3], "dependency": -2, "degree"= : 1.000000}, + + | {"attributes": [2, -1], "dependency": 3, "degree"= : 1.000000}, + + | {"attributes": [2, -1], "dependency": -2, "degree= ": 1.000000}, + + | {"attributes": [2, -2], "dependency": 3, "degree"= : 1.000000}, + + | {"attributes": [2, -2], "dependency": -1, "degree= ": 1.000000}, + + | {"attributes": [3, -1], "dependency": 2, "degree"= : 1.000000}, + + | {"attributes": [3, -1], "dependency": -2, "degree= ": 1.000000}, + + | {"attributes": [3, -2], "dependency": 2, "degree"= : 1.000000}, + + | {"attributes": [3, -2], "dependency": -1, "degree= ": 1.000000}, + + | {"attributes": [-1, -2], "dependency": 2, "degree= ": 0.500000}, + + | {"attributes": [-1, -2], "dependency": 3, "degree= ": 0.500000}, + + | {"attributes": [2, 3, -2], "dependency": -1, "deg= ree": 1.000000},+ + | {"attributes": [2, -1, -2], "dependency": 3, "deg= ree": 1.000000},+ + | {"attributes": [3, -1, -2], "dependency": 2, "deg= ree": 1.000000}] +most_common_vals |=20 +most_common_val_nulls |=20 +most_common_freqs |=20 +most_common_base_freqs |=20 + +-- if any one mcv param specified, all four must be specified (part 1) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 3) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 4) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[= ], + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[], + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[], + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + replace(e.dependencies, '}, ', E'},\n') AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+--------------------------------------------------= ---------------------------------------------------------------------------= --------------------------------------------------------------- +n_distinct | [{"attributes": [2, 3], "ndistinct": 4}, = = + + | {"attributes": [2, -1], "ndistinct": 4}, = = + + | {"attributes": [3, -1], "ndistinct": 4}, = = + + | {"attributes": [3, -2], "ndistinct": 4}, = = + + | {"attributes": [-1, -2], "ndistinct": 3}, = = + + | {"attributes": [2, 3, -1], "ndistinct": 4}, = = + + | {"attributes": [2, 3, -2], "ndistinct": 4}, = = + + | {"attributes": [2, -1, -2], "ndistinct": 4}, = = + + | {"attributes": [2, 3, -1, -2], "ndistinct": 4}] +dependencies | [{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, = + + | {"attributes": [2], "dependency": -1, "degree": 1= =2E000000}, = + + | {"attributes": [2], "dependency": -2, "degree": 1= =2E000000}, = + + | {"attributes": [3], "dependency": 2, "degree": 1.= 000000}, = + + | {"attributes": [3], "dependency": -1, "degree": 1= =2E000000}, = + + | {"attributes": [3], "dependency": -2, "degree": 1= =2E000000}, = + + | {"attributes": [-1], "dependency": 2, "degree": 0= =2E500000}, = + + | {"attributes": [-1], "dependency": 3, "degree": 0= =2E500000}, = + + | {"attributes": [-1], "dependency": -2, "degree": = 1.000000}, = + + | {"attributes": [-2], "dependency": 2, "degree": 0= =2E500000}, = + + | {"attributes": [-2], "dependency": 3, "degree": 0= =2E500000}, = + + | {"attributes": [-2], "dependency": -1, "degree": = 1.000000}, = + + | {"attributes": [2, 3], "dependency": -1, "degree"= : 1.000000}, = + + | {"attributes": [2, 3], "dependency": -2, "degree"= : 1.000000}, = + + | {"attributes": [2, -1], "dependency": 3, "degree"= : 1.000000}, = + + | {"attributes": [2, -1], "dependency": -2, "degree= ": 1.000000}, = + + | {"attributes": [2, -2], "dependency": 3, "degree"= : 1.000000}, = + + | {"attributes": [2, -2], "dependency": -1, "degree= ": 1.000000}, = + + | {"attributes": [3, -1], "dependency": 2, "degree"= : 1.000000}, = + + | {"attributes": [3, -1], "dependency": -2, "degree= ": 1.000000}, = + + | {"attributes": [3, -2], "dependency": 2, "degree"= : 1.000000}, = + + | {"attributes": [3, -2], "dependency": -1, "degree= ": 1.000000}, = + + | {"attributes": [-1, -2], "dependency": 2, "degree= ": 0.500000}, = + + | {"attributes": [-1, -2], "dependency": 3, "degree= ": 0.500000}, = + + | {"attributes": [2, 3, -2], "dependency": -1, "deg= ree": 1.000000}, = + + | {"attributes": [2, -1, -2], "dependency": 3, "deg= ree": 1.000000}, = + + | {"attributes": [3, -1, -2], "dependency": 2, "deg= ree": 1.000000}] +most_common_vals | {{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-2001,\= "{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-1,3}= ,{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}} +most_common_val_nulls | {{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}} +most_common_freqs | {0.25,0.25,0.25,0.25} +most_common_base_freqs | {0.00390625,0.015625,0.00390625,0.015625} + +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL,NULL,NULL},= {0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[] + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + e.inherited, e.null_frac, e.avg_width, e.n_distinct, e.most_common_val= s, + e.most_common_freqs, e.histogram_bounds, e.correlation, + e.most_common_elems, e.most_common_elem_freqs, e.elem_count_histogram +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +and e.inherited =3D false +\gx +-[ RECORD 1 ]----------+------- +inherited | f +null_frac | 0 +avg_width | 4 +n_distinct | -0.75 +most_common_vals | {1} +most_common_freqs | {0.5} +histogram_bounds | {-1,0} +correlation | -0.6 +most_common_elems |=20 +most_common_elem_freqs |=20 +elem_count_histogram |=20 +-[ RECORD 2 ]----------+------- +inherited | f +null_frac | 0.25 +avg_width | 4 +n_distinct | -0.5 +most_common_vals | {2} +most_common_freqs | {0.5} +histogram_bounds |=20 +correlation | 1 +most_common_elems |=20 +most_common_elem_freqs |=20 +elem_count_histogram |=20 + +SELECT + pg_catalog.pg_clear_extended_stats( + statistics_schemaname =3D> 'stats_import', + statistics_name =3D> 'test_stat_clone', + inherited =3D> false); + pg_clear_extended_stats=20 +------------------------- +=20 +(1 row) + +SELECT COUNT(*) +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + count=20 +------- + 0 +(1 row) + +SELECT COUNT(*) +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + count=20 +------- + 0 +(1 row) + +-- +-- Copy stats from test_stat to test_stat_clone +-- +SELECT + e.statistics_name, + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', e.statistics_schemaname::text, + 'statistics_name', 'test_stat_clone', + 'inherited', e.inherited, + 'n_distinct', e.n_distinct, + 'dependencies', e.dependencies, + 'most_common_vals', e.most_common_vals, + 'most_common_val_nulls', e.most_common_val_nulls, + 'most_common_freqs', e.most_common_freqs, + 'most_common_base_freqs', e.most_common_base_freqs, + 'exprs', x.exprs + ) +FROM pg_stats_ext AS e +CROSS JOIN LATERAL ( + SELECT + array_agg( + ARRAY[ee.null_frac::text, ee.avg_width::text, + ee.n_distinct::text, ee.most_common_vals::text, + ee.most_common_freqs::text, ee.histogram_bounds::text, + ee.correlation::text, ee.most_common_elems::text, + ee.most_common_elem_freqs::text, + ee.elem_count_histogram::text]) + FROM pg_stats_ext_exprs AS ee + WHERE ee.statistics_schemaname =3D e.statistics_schemaname + AND ee.statistics_name =3D e.statistics_name + AND ee.inherited =3D e.inherited + ) AS x(exprs) +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat'; + statistics_name | pg_restore_extended_stats=20 +-----------------+--------------------------- + test_stat | t +(1 row) + +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + inherited | n_distinct | dependencies | most_common_vals | most_common_va= l_nulls | most_common_freqs | most_common_base_freqs=20 +-----------+------------+--------------+------------------+---------------= --------+-------------------+------------------------ +(0 rows) + +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + inherited | n_distinct | dependencies | most_common_vals | most_common_va= l_nulls | most_common_freqs | most_common_base_freqs=20 +-----------+------------+--------------+------------------+---------------= --------+-------------------+------------------------ +(0 rows) + +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + inherited | null_frac | avg_width | n_distinct | most_common_vals | most_= common_freqs | histogram_bounds | correlation | most_common_elems | most_co= mmon_elem_freqs | elem_count_histogram=20 +-----------+-----------+-----------+------------+------------------+------= -------------+------------------+-------------+-------------------+--------= ----------------+---------------------- +(0 rows) + +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + inherited | null_frac | avg_width | n_distinct | most_common_vals | most_= common_freqs | histogram_bounds | correlation | most_common_elems | most_co= mmon_elem_freqs | elem_count_histogram=20 +-----------+-----------+-----------+------------+------------------+------= -------------+------------------+-------------+-------------------+--------= ----------------+---------------------- +(0 rows) + DROP SCHEMA stats_import CASCADE; NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to type stats_import.complex_type diff --git a/src/test/regress/sql/stats_import.sql b/src/test/regress/sql/s= tats_import.sql index d140733a7502..74700554c9c6 100644 --- a/src/test/regress/sql/stats_import.sql +++ b/src/test/regress/sql/stats_import.sql @@ -766,6 +766,9 @@ SELECT 4, 'four', NULL, int4range(0,100), NULL; =20 CREATE INDEX is_odd ON stats_import.test(((comp).a % 2 =3D 1)); =20 +CREATE STATISTICS stats_import.test_stat ON name, comp, lower(arange), arr= ay_length(tags,1) +FROM stats_import.test; + -- Generate statistics on table with data ANALYZE stats_import.test; =20 @@ -774,6 +777,9 @@ CREATE TABLE stats_import.test_clone ( LIKE stats_impor= t.test ) =20 CREATE INDEX is_odd_clone ON stats_import.test_clone(((comp).a % 2 =3D 1)); =20 +CREATE STATISTICS stats_import.test_stat_clone ON name, comp, lower(arange= ), array_length(tags,1) +FROM stats_import.test_clone; + -- -- Copy stats from test to test_clone, and is_odd to is_odd_clone -- @@ -970,4 +976,362 @@ AND tablename =3D 'stats_temp' AND inherited =3D false AND attname =3D 'i'; DROP TABLE stats_temp; + +-- set n_distinct using at attnum (1) that is not in the statistics object +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2], "ndistinct" : 4}, + {"attributes" : [1,2,3,-1,-2], "ndistinct" : 4}]':= :pg_ndistinct + ); + +-- set n_distinct using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [0,2,3,-1,-2], "ndistinct" : 4}]':= :pg_ndistinct + ); + +-- set n_distinct using at attnum that is outside the expression bounds(be= low -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [3,-1,-4], "ndistinct" : 4}, + {"attributes" : [3,-2,-4], "ndistinct" : 4}, + {"attributes" : [-1,-2,-4], "ndistinct" : 4}, + {"attributes" : [3,-1,-2,-4], "ndistinct" : 4}]'::= pg_ndistinct + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2], "ndistinct" : 4}]'::p= g_ndistinct + ); + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + e.dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +-- set dependencies using at attnum (1) that is not in the statistics obje= ct +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 1, "degree": 1= =2E000000}]'::pg_dependencies + ); + +-- set dependencies using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [0], "dependency": -1, "degree": = 1.000000}]'::pg_dependencies + ); + +-- set dependencies using at attnum that is outside the expression bounds(= below -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": -3, "degree": = 1.000000}]'::pg_dependencies + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, + {"attributes": [2], "dependency": -1, "degree": = 1.000000}, + {"attributes": [2], "dependency": -2, "degree": = 1.000000}, + {"attributes": [3], "dependency": 2, "degree": 1= =2E000000}, + {"attributes": [3], "dependency": -1, "degree": = 1.000000}, + {"attributes": [3], "dependency": -2, "degree": = 1.000000}, + {"attributes": [-1], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-1], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-1], "dependency": -2, "degree":= 1.000000}, + {"attributes": [-2], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-2], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-2], "dependency": -1, "degree":= 1.000000}, + {"attributes": [2,3], "dependency": -1, "degree"= : 1.000000}, + {"attributes": [2,3], "dependency": -2, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [2,-2], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [3,-1], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [3,-2], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [-1,-2], "dependency": 2, "degree= ": 0.500000}, + {"attributes": [-1,-2], "dependency": 3, "degree= ": 0.500000}, + {"attributes": [2,3,-2], "dependency": -1, "degr= ee": 1.000000}, + {"attributes": [2,-1,-2], "dependency": 3, "degr= ee": 1.000000}, + {"attributes": [3,-1,-2], "dependency": 2, "degr= ee": 1.000000}]'::pg_dependencies + ); + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + replace(e.dependencies, '}, ', E'},\n') AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +-- if any one mcv param specified, all four must be specified (part 1) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[] + ); + +-- if any one mcv param specified, all four must be specified (part 2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[] + ); + +-- if any one mcv param specified, all four must be specified (part 3) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[] + ); + +-- if any one mcv param specified, all four must be specified (part 4) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[= ], + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[], + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[], + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + +SELECT + replace(e.n_distinct, '}, ', E'},\n') AS n_distinct, + replace(e.dependencies, '}, ', E'},\n') AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL,NULL,NULL},= {0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[] + ); + +SELECT + e.inherited, e.null_frac, e.avg_width, e.n_distinct, e.most_common_val= s, + e.most_common_freqs, e.histogram_bounds, e.correlation, + e.most_common_elems, e.most_common_elem_freqs, e.elem_count_histogram +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +and e.inherited =3D false +\gx + +SELECT + pg_catalog.pg_clear_extended_stats( + statistics_schemaname =3D> 'stats_import', + statistics_name =3D> 'test_stat_clone', + inherited =3D> false); + +SELECT COUNT(*) +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + +SELECT COUNT(*) +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + +-- +-- Copy stats from test_stat to test_stat_clone +-- +SELECT + e.statistics_name, + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', e.statistics_schemaname::text, + 'statistics_name', 'test_stat_clone', + 'inherited', e.inherited, + 'n_distinct', e.n_distinct, + 'dependencies', e.dependencies, + 'most_common_vals', e.most_common_vals, + 'most_common_val_nulls', e.most_common_val_nulls, + 'most_common_freqs', e.most_common_freqs, + 'most_common_base_freqs', e.most_common_base_freqs, + 'exprs', x.exprs + ) +FROM pg_stats_ext AS e +CROSS JOIN LATERAL ( + SELECT + array_agg( + ARRAY[ee.null_frac::text, ee.avg_width::text, + ee.n_distinct::text, ee.most_common_vals::text, + ee.most_common_freqs::text, ee.histogram_bounds::text, + ee.correlation::text, ee.most_common_elems::text, + ee.most_common_elem_freqs::text, + ee.elem_count_histogram::text]) + FROM pg_stats_ext_exprs AS ee + WHERE ee.statistics_schemaname =3D e.statistics_schemaname + AND ee.statistics_name =3D e.statistics_name + AND ee.inherited =3D e.inherited + ) AS x(exprs) +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat'; + +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + DROP SCHEMA stats_import CASCADE; diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-adm= in.sgml index 2896cd9e4290..7fba7a833bdd 100644 --- a/doc/src/sgml/func/func-admin.sgml +++ b/doc/src/sgml/func/func-admin.sgml @@ -2165,6 +2165,104 @@ SELECT pg_restore_attribute_stats( + + + + pg_restore_extended_stats + + pg_restore_extended_stats ( + VARIADIC kwargs "a= ny" ) + boolean + + + Creates or updates statistics for statistics objects. Ordinarily, + these statistics are collected automatically or updated as a part= of + or = , so + it's not necessary to call this function. However, it is useful + after a restore to enable the optimizer to choose better plans if + ANALYZE has not been run yet. + + + The tracked statistics may change from version to version, so + arguments are passed as pairs of argname + and argvalue in the form: + + SELECT pg_restore_extended_stats( + 'arg1name', 'arg1value'::arg1type, + 'arg2name', 'arg2value'::arg2type, + 'arg3name', 'arg3value'::arg3type); + + + + For example, to set the n_distinct, + dependencies, and exprs + values for the statistics object myschema.mystatsobj<= /structname>: + + SELECT pg_restore_extended_stats( + 'statistics_schemaname', 'myschema'::name, + 'statistics_name', 'mytable'::name, + 'inherited', false, + 'n_distinct', '{"2, 3": 4, "2, -1": 4, "2, -2": 4, "3, -1":= 4, "3, -2": 4}'::pg_ndistinct, + 'dependencies', '{"2 =3D> 1": 1.000000, "2 =3D> -1": 1.000000= , "2 =3D> -2": 1.000000}'::pg_dependencies + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL= ,NULL,NULL},{0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[]); + + + + The required arguments are statistics_schemaname with a value + of type name, which specifies the statistics object'= s schema; + statistics_name with a value of type nam= e, which specifies + the name of the statistics object; and inherited, which + specifies whether the statistics include values from child tables. + Other arguments are the names and values of statistics correspond= ing + to columns in pg_= stats_ext + . To accept statistics for any expressions in the extended= statistics object, the + parameter exprs with a type text[] is available, the array + must be two dimensional with an outer array in length equal to th= e number of expressions in + the object, and the inner array elements for each of the statisti= cal columns in pg_stats_ext_expr= s, some + of which are themselves arrays. + + + Additionally, this function accepts argument name + version of type integer, which + specifies the server version from which the statistics originated. + This is anticipated to be helpful in porting statistics from older + versions of PostgreSQL. + + + Minor errors are reported as a WARNING and + ignored, and remaining statistics will still be restored. If all + specified statistics are successfully restored, returns + true, otherwise false. + + + The caller must have the MAINTAIN privilege on= the + table or be the owner of the database. + + + + + + + + pg_clear_extended_stats + + pg_clear_extended_stats ( + statistics_schemaname name, + statistics_name name, + inherited boolean ) + void + + + Clears statistics for the given statistics object, as + though the object was newly created. + + + The caller must have the MAINTAIN privilege on + the table or be the owner of the database. + + + --=20 2.51.0 --mcHHJuWZHCX7IVwX Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v23-0002-Include-Extended-Statistics-in-pg_dump.patch" Content-Transfer-Encoding: quoted-printable =46rom 8574e374cec4dd30e345f23a07633eab6160f6c4 Mon Sep 17 00:00:00 2001 =46rom: Corey Huinker Date: Wed, 5 Nov 2025 01:13:04 -0500 Subject: [PATCH v23 2/2] Include Extended Statistics in pg_dump. Incorporate the new pg_restore_extended_stats() function into pg_dump. This detects the existence of extended statistics statistics (i.e. pg_statistic_ext_data rows). This handles many of the changes that have happened to extended statistic statistics over the various versions, including: * Format change for pg_ndistinct and pg_dependencies in current development version. Earlier versions have the format translated via the pg_dump SQL statement. * Inherited extended statistics were introduced in v15. * Expressions were introduced to extended statistics in v14. * MCV extended statistics were introduced in v13. * pg_statistic_ext_data and pg_stats_ext introduced in v12, prior to that ndstinct and depdendencies data (the only kind of stats that existed were directly on pg_statistic_ext. * Extended Statistics were introduced in v10, so there is no support for prior versions necessary. --- src/bin/pg_dump/pg_backup.h | 1 + src/bin/pg_dump/pg_backup_archiver.c | 3 +- src/bin/pg_dump/pg_dump.c | 254 +++++++++++++++++++++++++++ src/bin/pg_dump/t/002_pg_dump.pl | 28 +++ 4 files changed, 285 insertions(+), 1 deletion(-) diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h index d9041dad7206..2f8d9799c30c 100644 --- a/src/bin/pg_dump/pg_backup.h +++ b/src/bin/pg_dump/pg_backup.h @@ -68,6 +68,7 @@ enum _dumpPreparedQueries PREPQUERY_DUMPCOMPOSITETYPE, PREPQUERY_DUMPDOMAIN, PREPQUERY_DUMPENUMTYPE, + PREPQUERY_DUMPEXTSTATSOBJSTATS, PREPQUERY_DUMPFUNC, PREPQUERY_DUMPOPR, PREPQUERY_DUMPRANGETYPE, diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_back= up_archiver.c index 4a63f7392ae8..1d42b0343eb3 100644 --- a/src/bin/pg_dump/pg_backup_archiver.c +++ b/src/bin/pg_dump/pg_backup_archiver.c @@ -3007,7 +3007,8 @@ _tocEntryRequired(TocEntry *te, teSection curSection,= ArchiveHandle *AH) strcmp(te->desc, "SEARCHPATH") =3D=3D 0) return REQ_SPECIAL; =20 - if (strcmp(te->desc, "STATISTICS DATA") =3D=3D 0) + if ((strcmp(te->desc, "STATISTICS DATA") =3D=3D 0) || + (strcmp(te->desc, "EXTENDED STATISTICS DATA") =3D=3D 0)) { if (!ropt->dumpStatistics) return 0; diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 27f6be3f0f82..d7e6c71f140d 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -324,6 +324,7 @@ static void dumpSequenceData(Archive *fout, const Table= DataInfo *tdinfo); static void dumpIndex(Archive *fout, const IndxInfo *indxinfo); static void dumpIndexAttach(Archive *fout, const IndexAttachInfo *attachin= fo); static void dumpStatisticsExt(Archive *fout, const StatsExtInfo *statsexti= nfo); +static void dumpStatisticsExtStats(Archive *fout, const StatsExtInfo *stat= sextinfo); static void dumpConstraint(Archive *fout, const ConstraintInfo *coninfo); static void dumpTableConstraintComment(Archive *fout, const ConstraintInfo= *coninfo); static void dumpTSParser(Archive *fout, const TSParserInfo *prsinfo); @@ -8272,6 +8273,9 @@ getExtendedStatistics(Archive *fout) =20 /* Decide whether we want to dump it */ selectDumpableStatisticsObject(&(statsextinfo[i]), fout); + + if (fout->dopt->dumpStatistics) + statsextinfo[i].dobj.components |=3D DUMP_COMPONENT_STATISTICS; } =20 PQclear(res); @@ -11726,6 +11730,7 @@ dumpDumpableObject(Archive *fout, DumpableObject *d= obj) break; case DO_STATSEXT: dumpStatisticsExt(fout, (const StatsExtInfo *) dobj); + dumpStatisticsExtStats(fout, (const StatsExtInfo *) dobj); break; case DO_REFRESH_MATVIEW: refreshMatViewData(fout, (const TableDataInfo *) dobj); @@ -18528,6 +18533,255 @@ dumpStatisticsExt(Archive *fout, const StatsExtIn= fo *statsextinfo) free(qstatsextname); } =20 +/* + * dumpStatisticsExtStats + * write out to fout the stats for an extended statistics object + */ +static void +dumpStatisticsExtStats(Archive *fout, const StatsExtInfo *statsextinfo) +{ + DumpOptions *dopt =3D fout->dopt; + PQExpBuffer query; + PGresult *res; + int nstats; + + /* Do nothing if not dumping statistics */ + if (!dopt->dumpStatistics) + return; + + if (!fout->is_prepared[PREPQUERY_DUMPEXTSTATSOBJSTATS]) + { + PQExpBuffer pq =3D createPQExpBuffer(); + + /* + * Set up query for constraint-specific details. + * + * 19+: query pg_stats_ext and pg_stats_ext_exprs as-is 15-18: query + * pg_stats_ext translating the ndistinct and dependencies, 14: + * inherited is always NULL 12-13: no pg_stats_ext_exprs 10-11: no + * pg_stats_ext, join pg_statistic_ext and pg_namespace + */ + + appendPQExpBufferStr(pq, + "PREPARE getExtStatsStats(pg_catalog.name, pg_catalog.name) AS\n" + "SELECT "); + + /* + * Versions 15+ have inherited stats. + * + * Create this column in all version because we need to order by it + * later. + */ + if (fout->remoteVersion >=3D 150000) + appendPQExpBufferStr(pq, "e.inherited, "); + else + appendPQExpBufferStr(pq, "false AS inherited, "); + + /* + * The ndistinnct and depdendencies formats changed in v19, so + * everything before that needs to be translated. + * + * The ndistinct translation converts this: + * + * {"3, 4": 11, "3, 6": 11, "4, 6": 11, "3, 4, 6": 11} + * + * to this: + * + * [ {"attributes": [3,4], "ndistinct": 11}, {"attributes": [3,6], + * "ndistinct": 11}, {"attributes": [4,6], "ndistinct": 11}, + * {"attributes": [3,4,6], "ndistinct": 11} ] + * + * and the dependencies translation converts this: + * + * {"3 =3D> 4": 1.000000, "3 =3D> 6": 1.000000, "4 =3D> 6": 1.000000, "3= , 4 + * =3D> 6": 1.000000, "3, 6 =3D> 4": 1.000000} + * + * to this: + * + * [ {"attributes": [3], "dependency": 4, "degree": 1.000000}, + * {"attributes": [3], "dependency": 6, "degree": 1.000000}, + * {"attributes": [4], "dependency": 6, "degree": 1.000000}, + * {"attributes": [3,4], "dependency": 6, "degree": 1.000000}, + * {"attributes": [3,6], "dependency": 4, "degree": 1.000000} ] + */ + if (fout->remoteVersion >=3D 190000) + appendPQExpBufferStr(pq, "e.n_distinct, e.dependencies, "); + else + appendPQExpBufferStr(pq, + "( " + "SELECT json_agg( " + " json_build_object( " + " 'attributes', " + " string_to_array(kv.key, ', ')::integer[], " + " 'ndistinct', " + " kv.value::bigint )) " + "FROM json_each_text(e.n_distinct::text::json) AS kv" + ") AS n_distinct, " + "( " + "SELECT json_agg( " + " json_build_object( " + " 'attributes', " + " string_to_array( " + " split_part(kv.key, ' =3D> ', 1), " + " ', ')::integer[], " + " 'dependency', " + " split_part(kv.key, ' =3D> ', 2)::integer, " + " 'degree', " + " kv.value::double precision )) " + "FROM json_each_text(e.dependencies::text::json) AS kv " + ") AS dependencies, "); + + /* MCV was introduced v13 */ + if (fout->remoteVersion >=3D 130000) + appendPQExpBufferStr(pq, + "e.most_common_vals, e.most_common_val_nulls, " + "e.most_common_freqs, e.most_common_base_freqs, "); + else + appendPQExpBufferStr(pq, + "NULL AS most_common_vals, NULL AS most_common_val_nulls, " + "NULL AS most_common_freqs, NULL AS most_common_base_freqs, "); + + /* Expressions were introduced in v14 */ + if (fout->remoteVersion >=3D 140000) + { + appendPQExpBufferStr(pq, + "( " + "SELECT array_agg( " + " ARRAY[ee.null_frac::text, ee.avg_width::text, " + " ee.n_distinct::text, ee.most_common_vals::text, " + " ee.most_common_freqs::text, ee.histogram_bounds::text,= " + " ee.correlation::text, ee.most_common_elems::text, " + " ee.most_common_elem_freqs::text, " + " ee.elem_count_histogram::text]) " + "FROM pg_stats_ext_exprs AS ee " + "WHERE ee.statistics_schemaname =3D $1 " + "AND ee.statistics_name =3D $2 "); + + /* Inherited expressions introduced in v15 */ + if (fout->remoteVersion >=3D 150000) + appendPQExpBufferStr(pq, "AND ee.inherited =3D e.inherited"); + + appendPQExpBufferStr(pq, ") AS exprs "); + } + else + appendPQExpBufferStr(pq, "NULL AS exprs "); + + /* pg_stats_ext introduced in v12 */ + if (fout->remoteVersion >=3D 120000) + appendPQExpBufferStr(pq, + "FROM pg_catalog.pg_stats_ext AS e " + "WHERE e.statistics_schemaname =3D $1 " + "AND e.statistics_name =3D $2 "); + else + appendPQExpBufferStr(pq, + "FROM ( " + "SELECT s.stxndistinct AS n_distinct, " + " s.stxdependencies AS dependencies " + "FROM pg_catalog.pg_statistics_ext AS s " + "JOIN pg_catalog.pg_namespace AS n " + "ON n.oid =3D s.stxnamespace " + "WHERE n.nspname =3D $1 " + "AND e.stxname =3D $2 " + ") AS e "); + + /* we always have an inherited column, but it may be a constant */ + appendPQExpBufferStr(pq, "ORDER BY inherited"); + + ExecuteSqlStatement(fout, pq->data); + + fout->is_prepared[PREPQUERY_DUMPEXTSTATSOBJSTATS] =3D true; + + destroyPQExpBuffer(pq); + } + + query =3D createPQExpBuffer(); + + appendPQExpBufferStr(query, "EXECUTE getExtStatsStats("); + appendStringLiteralAH(query, statsextinfo->dobj.namespace->dobj.name, fou= t); + appendPQExpBufferStr(query, "::pg_catalog.name, "); + appendStringLiteralAH(query, statsextinfo->dobj.name, fout); + appendPQExpBufferStr(query, "::pg_catalog.name)"); + + res =3D ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + destroyPQExpBuffer(query); + + nstats =3D PQntuples(res); + + if (nstats > 0) + { + PQExpBuffer out =3D createPQExpBuffer(); + + int i_inherited =3D PQfnumber(res, "inherited"); + int i_ndistinct =3D PQfnumber(res, "n_distinct"); + int i_dependencies =3D PQfnumber(res, "dependencies"); + int i_mcv =3D PQfnumber(res, "most_common_vals"); + int i_mcv_nulls =3D PQfnumber(res, "most_common_val_nulls"); + int i_mcf =3D PQfnumber(res, "most_common_freqs"); + int i_mcbf =3D PQfnumber(res, "most_common_base_freqs"); + int i_exprs =3D PQfnumber(res, "exprs"); + + for (int i =3D 0; i < nstats; i++) + { + if (PQgetisnull(res, i, i_inherited)) + pg_fatal("inherited cannot be NULL"); + + appendPQExpBufferStr(out, + "SELECT * FROM pg_catalog.pg_restore_extended_stats(\n"); + appendPQExpBuffer(out, "\t'version', '%d'::integer,\n", + fout->remoteVersion); + appendPQExpBufferStr(out, "\t'statistics_schemaname', "); + appendStringLiteralAH(out, statsextinfo->dobj.namespace->dobj.name, fou= t); + appendPQExpBufferStr(out, ",\n\t'statistics_name', "); + appendStringLiteralAH(out, statsextinfo->dobj.name, fout); + appendNamedArgument(out, fout, "inherited", "boolean", + PQgetvalue(res, i, i_inherited)); + + if (!PQgetisnull(res, i, i_ndistinct)) + appendNamedArgument(out, fout, "n_distinct", "pg_ndistinct", + PQgetvalue(res, i, i_ndistinct)); + + if (!PQgetisnull(res, i, i_dependencies)) + appendNamedArgument(out, fout, "dependencies", "pg_dependencies", + PQgetvalue(res, i, i_dependencies)); + + if (!PQgetisnull(res, i, i_mcv)) + appendNamedArgument(out, fout, "most_common_vals", "text[]", + PQgetvalue(res, i, i_mcv)); + + if (!PQgetisnull(res, i, i_mcv_nulls)) + appendNamedArgument(out, fout, "most_common_val_nulls", "boolean[]", + PQgetvalue(res, i, i_mcv_nulls)); + + if (!PQgetisnull(res, i, i_mcf)) + appendNamedArgument(out, fout, "most_common_freqs", "double precision[= ]", + PQgetvalue(res, i, i_mcf)); + + if (!PQgetisnull(res, i, i_mcbf)) + appendNamedArgument(out, fout, "most_common_base_freqs", "double preci= sion[]", + PQgetvalue(res, i, i_mcbf)); + + if (!PQgetisnull(res, i, i_exprs)) + appendNamedArgument(out, fout, "exprs", "text[]", + PQgetvalue(res, i, i_exprs)); + + appendPQExpBufferStr(out, "\n);\n"); + } + + ArchiveEntry(fout, nilCatalogId, createDumpId(), + ARCHIVE_OPTS(.tag =3D statsextinfo->dobj.name, + .namespace =3D statsextinfo->dobj.namespace->dobj.name, + .owner =3D statsextinfo->rolname, + .description =3D "EXTENDED STATISTICS DATA", + .section =3D SECTION_POST_DATA, + .createStmt =3D out->data, + .deps =3D &statsextinfo->dobj.dumpId, + .nDeps =3D 1)); + destroyPQExpBuffer(out); + } + PQclear(res); +} + /* * dumpConstraint * write out to fout a user-defined constraint diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_du= mp.pl index e33aa95f6ffc..9ab82f972773 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -4772,6 +4772,34 @@ my %tests =3D ( }, }, =20 + # + # EXTENDED stats will end up in SECTION_POST_DATA. + # + 'extended_statistics_import' =3D> { + create_sql =3D> ' + CREATE TABLE dump_test.has_ext_stats + AS SELECT g.g AS x, g.g / 2 AS y FROM generate_series(1,100) AS g(g); + CREATE STATISTICS dump_test.es1 ON x, (y % 2) FROM dump_test.has_ext_st= ats; + ANALYZE dump_test.has_ext_stats;', + regexp =3D> qr/^ + \QSELECT * FROM pg_catalog.pg_restore_extended_stats(\E\s+/xm, + like =3D> { + %full_runs, + %dump_test_schema_runs, + no_data_no_schema =3D> 1, + no_schema =3D> 1, + section_post_data =3D> 1, + statistics_only =3D> 1, + schema_only_with_statistics =3D> 1, + }, + unlike =3D> { + exclude_dump_test_schema =3D> 1, + no_statistics =3D> 1, + only_dump_measurement =3D> 1, + schema_only =3D> 1, + }, + }, + # # While attribute stats (aka pg_statistic stats) only appear for tables # that have been analyzed, all tables will have relation stats because --=20 2.51.0 --mcHHJuWZHCX7IVwX-- --qEAULHxEpVnoWmAO Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmlM2H4ACgkQnvQgOdby QH3dbQ/+NMSOzJoFyHnX3YeZEmg8tgAoYudrvEbtTZcn8XjUD2BjtNYZQ6GUjmK4 QdNBsXwMkHdqX+LArqQfX/QOsy2MfzPCz4mrqHG6+cfdLWo8R+30L8si6x427ypq ijKrQ00tVfolVllEpQ08MsyH46SK4TGNeEPNlyVMAeV6edwO5qsGtZx3TWH5UMDE 7OhckeEUV6s0wguMkGiJMDRMRqb8hlKuTclk6vVbBYNh81VLT7YV98thETkPyzNa sihx7RZGx88UCvKoAWgXNjenH7GOKyf1OjG8694pyJrxIoldyezxZzSQ23z2OHDS 6PpI1keTxRZ7/brhn8Q+phFnLcza3FB6ttGGgsop+KbqG5cxhmbnuJp0ZkgyYjv9 P/6MnllpQGgpE3HQxLpaxnfipnJ8BwZB39Mmu+iPdenghB0dBO8jTDYPFtK0iW1d eTQ5buroYH9jLaOG/yA7Lv1h+t1ZiT1o11i/2HYa/zrGad0669aGnLuBN7+gYiot TA/nfIUiztHr+sGIFL0JG9VImXbaO1CSAnQuJrSEUsVJs6w49OT14zMBFMmbzikC OBjyDN5GO9Y26ac6Cp/fsM1kVRr3eV6rq6Yf2D2ShXpnRya4w4DV8EzxfHmTh4nR Hjm13JBdEFbL1x288NzCs1dDgJ2rUth/6fxTkit6SxLozIZxzMI= =okRJ -----END PGP SIGNATURE----- --qEAULHxEpVnoWmAO--