Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vKtAN-00CMmH-0T for pgsql-hackers@arkaria.postgresql.org; Mon, 17 Nov 2025 06:56:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vKtAJ-00CWxU-3C for pgsql-hackers@arkaria.postgresql.org; Mon, 17 Nov 2025 06:56:44 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vKtAI-00CWxJ-1m for pgsql-hackers@lists.postgresql.org; Mon, 17 Nov 2025 06:56:43 +0000 Received: from fhigh-a1-smtp.messagingengine.com ([103.168.172.152]) by makus.postgresql.org with smtp (Exim 4.96) (envelope-from ) id 1vKtAA-007WRp-2V for pgsql-hackers@lists.postgresql.org; Mon, 17 Nov 2025 06:56:41 +0000 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.phl.internal (Postfix) with ESMTP id C07BD140015E; Mon, 17 Nov 2025 01:56:33 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Mon, 17 Nov 2025 01:56:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1763362593; x=1763448993; bh=q1BsY0lZwT KSgFgPQ9ZOh7TaegN+Sqe8zeGY6kUcSy8=; b=KzCTC/fueX2SN2gGohx68uZe74 xf5DvviWPdYxGbmRncCQ1M+PxxvuRUf1JXvFwUClUthLHgiQr7HR2AeHKdd/bkEc gMRAqqA9EyuKJJ4yzFVOW9XnhKLDlB1D+VswhoFDsYNLmI5udLAUC0g5wg9kll66 qrV2f2i8qCNwsD6NAuri13ubL/Aj4X9ZUh/8mQ9IgMoUL7bjvWS7Y5c6RlKcLiZx GTF+qJxx1kPPFOAwWwctIf+VNhK54EBzeJtKwKEG7jbVNM/xVneJrEBqmfiU+7fG U6yNaKRT2sKOrqn6NINl9UoHzg7DeT6zcDoAcuEo3MlPwmOnZk51FJ1typ2g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1763362593; x=1763448993; bh=q1BsY0lZwTKSgFgPQ9ZOh7TaegN+Sqe8zeG Y6kUcSy8=; b=aTE0kH0IJBKVXkFPDz+ji8E/pnnBMkH/xnZ/W4AyMajckf+pUii i80LpIkfmXGK5H8yUBTuQwGfikSOJrcP145cBg9OTgF/ZOIH37/XwAhHj3LJvgX4 7l4m7P1MFQLveV36gaOJFfvLjktrfAkGEctsHYnDbRV8j5v2oZnACw4s5nuTkF4W n5p+Aj2VfGv0wY34tlL8yx2GuUXPHryGLFGnyWA9MefSCoJC1iPLBlwEVCVI9c7L BT4xLqyo155vZ9oq5HeigHs8pai8sLdtMgPvCxQocc3d9TYkEahvTNAXa4yVpG0+ kQrTKOfDvCaqptIc/IBlau0OZd/0WbGHc3g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggddvudejkedtucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfurfetoffkrfgpnffqhgenuceu rghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnegfrh hlucfvnfffucdljedtmdenucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddv necuhfhrohhmpefoihgthhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesphgrqh huihgvrhdrgiihiieqnecuggftrfgrthhtvghrnhepteelieefudffhffhtdetleeggeeg fffhkeeuveetiefgudduvedutefggeeivdejnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepmhhitghhrggvlhesphgrqhhuihgvrhdrgiihiidp nhgspghrtghpthhtohephedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoheptghorh gvhidrhhhuihhnkhgvrhesghhmrghilhdrtghomhdprhgtphhtthhopehjihgrnhdruhhn ihhvvghrshgrlhhithihsehgmhgrihhlrdgtohhmpdhrtghpthhtohepthhomhgrshesvh honhgurhgrrdhmvgdprhgtphhtthhopehpghhsqhhlqdhhrggtkhgvrhhssehlihhsthhs rdhpohhsthhgrhgvshhqlhdrohhrghdprhgtphhtthhopehtghhlsehsshhsrdhpghhhrd hprgdruhhs X-ME-Proxy: Feedback-ID: i0fe9450f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 17 Nov 2025 01:56:29 -0500 (EST) Date: Mon, 17 Nov 2025 15:56:06 +0900 From: Michael Paquier To: Corey Huinker Cc: jian he , Tomas Vondra , pgsql-hackers@lists.postgresql.org, tgl@sss.pgh.pa.us Subject: Re: Extended Statistics set/restore/clear functions. Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="eH7/90dwF2tDSx+7" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --eH7/90dwF2tDSx+7 Content-Type: multipart/mixed; boundary="AorqrXjCfzfo1Gz1" Content-Disposition: inline --AorqrXjCfzfo1Gz1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Nov 14, 2025 at 03:25:27PM +0900, Michael Paquier wrote: > Thanks for the new versions, I'll also look at all these across the > next couple of days. Probably not at 0005~ for now. 0001 and 0002 from series v13 have been applied to change the output functions. And I have looked at 0003 in details for now. Attached is a revised version for it, with many adjustments. Some notes: - Many portions of the coverage were missing. I have measured the coverage at 91% with the updated version attached. This includes coverage for some error reporting, something that we rely a lot on for this code. - The error reports are made simpler, with the token values getting hidden. While testing with some fancy values, I have actually noticed that the error handlings for the parsing of the int16 and int32 values were incorrect, the error reports used what the safe functions generated, not the reports from the data type. - Passing down arbitrary bytes sequences was leading to these bytes reported in the error outputs because we cared about the token values. I have added a few tests based on that for the code paths involved. There is an extra thing that bugs me as incorrect for the pg_ndistinct input, something I have not tackled myself yet. Your patch checks that subsets of attributes are included in the longest set found, but it does not match the guarantees we have in mvndistinct.c: we have to check that *all* the combinations generated by generator_init() are satisfied based on the longest of attributes detected. For example, this is thought as correct in the input function: SELECT '[{"attributes" : [-1,2], "ndistinct" : 1}, {"attributes" : [-1,2,3], "ndistinct" : 3}]'::pg_ndistinct; However it is obviously not correct as we are missing an element for the attributes [-1, 3]. The simplest solution would be to export the routines that generate the groups now in mvndistinct.c. Also we should make sure that the number of elements in the arrays match with the number of groups we expect, not only the elements. I don't think that we need to care much about the values, but we ought to provide stronger guarantees for the attributes listed in these elements. Except for this argument, the input of pg_ndistinct feels OK in terms of the guarantees that we'd want to enforce on an import. The same argument applies in terms of attribute number guarantees for pg_dependencies, based on DependencyGenerator_init() & friends in dependencies.c. Could you look at that? For pg_dependencies, we also require some checks on the value for "dependency", of course, making sure that this matches with what's expected with the "largest" sets of attributes. In this case, we need to track the union of "dependency" and "attributes", with "attributes" having at least one element. The tests of pg_dependencies need also to be extended more (begun that a bit, far from being complete and I'm lacking of time this week due to a conference). One thing that I would add are nested JSON objects in the paths where we expect values, for example. Please note that I have done a brush of 0004, while on it, cleaning up typos, inconsistencies and making the error codes consistent with the ndistinct case where possible. This is not ready, but that's at least it's a start to rely on. In terms of committable bits, it would be better to apply the input functions once both parts are ready to go. For now I am attached a v14 with the work I've put into them. 0005~ are not reviewed yet, as mentioned previously. The changes in pg_dependencies are actually straight-forward to figure out (well, mostly) once the pg_ndistinct changes are OK in shape. -- Michael --AorqrXjCfzfo1Gz1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v14-0001-Add-working-input-function-for-pg_ndistinct.patch" Content-Transfer-Encoding: quoted-printable =46rom 5bfc7c74ed90c122bc39874d99f6af38f2f944fa Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Mon, 17 Nov 2025 14:52:22 +0900 Subject: [PATCH v14 1/5] Add working input function for pg_ndistinct. This will consume the format that was established when the output function for pg_ndistinct was recently changed. This will be needed for importing extended statistics. With these changes in place, coverage of pg_ndistinct.c reaches 91%. --- src/backend/utils/adt/pg_ndistinct.c | 695 ++++++++++++++++++++- src/test/regress/expected/pg_ndistinct.out | 359 +++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/pg_ndistinct.sql | 87 +++ src/tools/pgindent/typedefs.list | 2 + 5 files changed, 1136 insertions(+), 9 deletions(-) create mode 100644 src/test/regress/expected/pg_ndistinct.out create mode 100644 src/test/regress/sql/pg_ndistinct.sql diff --git a/src/backend/utils/adt/pg_ndistinct.c b/src/backend/utils/adt/p= g_ndistinct.c index 97efc290ef5e..4c13b2e62885 100644 --- a/src/backend/utils/adt/pg_ndistinct.c +++ b/src/backend/utils/adt/pg_ndistinct.c @@ -14,27 +14,706 @@ =20 #include "postgres.h" =20 +#include "common/int.h" +#include "common/jsonapi.h" #include "lib/stringinfo.h" +#include "mb/pg_wchar.h" +#include "nodes/miscnodes.h" #include "statistics/extended_stats_internal.h" #include "statistics/statistics_format.h" +#include "utils/builtins.h" #include "utils/fmgrprotos.h" =20 +/* Parsing state data */ +typedef enum +{ + NDIST_EXPECT_START =3D 0, + NDIST_EXPECT_ITEM, + NDIST_EXPECT_KEY, + NDIST_EXPECT_ATTNUM_LIST, + NDIST_EXPECT_ATTNUM, + NDIST_EXPECT_NDISTINCT, + NDIST_EXPECT_COMPLETE +} NDistinctSemanticState; + +typedef struct +{ + const char *str; + NDistinctSemanticState state; + + List *distinct_items; /* Accumulated complete MVNDistinctItems */ + Node *escontext; + + bool found_attributes; /* Item has "attributes" key */ + bool found_ndistinct; /* Item has "ndistinct" key */ + List *attnum_list; /* Accumulated attributes attnums */ + int32 ndistinct; +} NDistinctParseState; + +/* + * Invoked at the start of each MVNDistinctItem. + * + * The entire JSON document should be one array of MVNDistinctItem objects. + * If we are anywhere else in the document, it is an error. + */ +static JsonParseErrorType +ndistinct_object_start(void *state) +{ + NDistinctParseState *parse =3D state; + + switch (parse->state) + { + case NDIST_EXPECT_ITEM: + /* Now we expect to see attributes/ndistinct keys */ + parse->state =3D NDIST_EXPECT_KEY; + return JSON_SUCCESS; + break; + + case NDIST_EXPECT_START: + /* pg_ndistinct must begin with a '[' */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Initial element must be an array.")); + break; + + case NDIST_EXPECT_KEY: + /* In an object, expecting key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Expected an object key.")); + break; + + case NDIST_EXPECT_ATTNUM_LIST: + /* Just followed an "attributes" key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Value of \"%s\" must be an array of attribute numbers.", + PG_NDISTINCT_KEY_ATTRIBUTES)); + break; + + case NDIST_EXPECT_ATTNUM: + /* In an attnum list, expect only scalar integers */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Attribute lists can only contain attribute numbers.")); + break; + + case NDIST_EXPECT_NDISTINCT: + /* Just followed an "ndistinct" key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Value of \"%s\" must be an integer.", + PG_NDISTINCT_KEY_NDISTINCT)); + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Unexpected parse state: %d", (int) parse->state)); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * Routine to allow qsorting of AttNumbers + */ +static int +attnum_compare(const void *aptr, const void *bptr) +{ + AttrNumber a =3D *(const AttrNumber *) aptr; + AttrNumber b =3D *(const AttrNumber *) bptr; + + return pg_cmp_s16(a, b); +} + =20 /* - * pg_ndistinct_in - * input routine for type pg_ndistinct + * Invoked at the end of an object. * - * pg_ndistinct is real enough to be a table column, but it has no - * operations of its own, and disallows input (just like pg_node_tree). + * Check to ensure that it was a complete MVNDistinctItem + */ +static JsonParseErrorType +ndistinct_object_end(void *state) +{ + NDistinctParseState *parse =3D state; + + int natts =3D 0; + AttrNumber *attrsort; + + MVNDistinctItem *item; + + if (parse->state !=3D NDIST_EXPECT_KEY) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Unexpected parse state: %d", (int) parse->state)); + return JSON_SEM_ACTION_FAILED; + } + + if (!parse->found_attributes) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Item must contain \"%s\" key.", + PG_NDISTINCT_KEY_ATTRIBUTES)); + return JSON_SEM_ACTION_FAILED; + } + + if (!parse->found_ndistinct) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Item must contain \"%s\" key.", + PG_NDISTINCT_KEY_NDISTINCT)); + return JSON_SEM_ACTION_FAILED; + } + + /* + * We need at least two attribute numbers for a ndistinct item, anything + * less is malformed. + */ + natts =3D parse->attnum_list->length; + if (natts < 2) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("The \"%s\" key must contain an array of at least two attrib= utes.", + PG_NDISTINCT_KEY_NDISTINCT)); + return JSON_SEM_ACTION_FAILED; + } + attrsort =3D palloc0(natts * sizeof(AttrNumber)); + + /* Create the MVNDistinctItem */ + item =3D palloc(sizeof(MVNDistinctItem)); + item->nattributes =3D natts; + item->attributes =3D palloc0(natts * sizeof(AttrNumber)); + item->ndistinct =3D (double) parse->ndistinct; + + /* fill out both attnum list and sortable list */ + for (int i =3D 0; i < natts; i++) + { + attrsort[i] =3D (AttrNumber) parse->attnum_list->elements[i].int_value; + item->attributes[i] =3D attrsort[i]; + } + + /* Check attrsort for uniqueness */ + qsort(attrsort, natts, sizeof(AttrNumber), attnum_compare); + for (int i =3D 1; i < natts; i++) + { + if (attrsort[i] =3D=3D attrsort[i - 1]) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Duplicated attribute number found: %d.", attrsort[i])); + pfree(attrsort); + pfree(item->attributes); + pfree(item); + return JSON_SEM_ACTION_FAILED; + } + } + pfree(attrsort); + + parse->distinct_items =3D lappend(parse->distinct_items, (void *) item); + + /* reset item state vars */ + list_free(parse->attnum_list); + parse->attnum_list =3D NIL; + parse->ndistinct =3D 0; + parse->found_attributes =3D false; + parse->found_ndistinct =3D false; + + /* Now we are looking for the next MVNDistinctItem */ + parse->state =3D NDIST_EXPECT_ITEM; + return JSON_SUCCESS; +} + + +/* + * ndistinct input format has two types of arrays, the outer MVNDistinctIt= em + * array and the attribute number array within each MVNDistinctItem. + */ +static JsonParseErrorType +ndistinct_array_start(void *state) +{ + NDistinctParseState *parse =3D state; + + switch (parse->state) + { + case NDIST_EXPECT_ATTNUM_LIST: + parse->state =3D NDIST_EXPECT_ATTNUM; + break; + + case NDIST_EXPECT_START: + parse->state =3D NDIST_EXPECT_ITEM; + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Array found in unexpected place.")); + return JSON_SEM_ACTION_FAILED; + } + + return JSON_SUCCESS; +} + + +/* + * Arrays can never be empty. + */ +static JsonParseErrorType +ndistinct_array_end(void *state) +{ + NDistinctParseState *parse =3D state; + + switch (parse->state) + { + case NDIST_EXPECT_ATTNUM: + if (parse->attnum_list !=3D NIL) + { + /* + * The attribute number list is complete, look for more + * MVNDistinctItem keys. + */ + parse->state =3D NDIST_EXPECT_KEY; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("The \"%s\" key must be an non-empty array.", + PG_NDISTINCT_KEY_ATTRIBUTES)); + break; + + case NDIST_EXPECT_ITEM: + if (parse->distinct_items !=3D NIL) + { + /* Item list is complete, we are done. */ + parse->state =3D NDIST_EXPECT_COMPLETE; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Item array cannot be empty.")); + break; + default: + + /* + * This can only happen if a case was missed in + * ndistinct_array_start(). + */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Array found in unexpected place.")); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * The valid keys for the MVNDistinctItem object are: + * - attributes + * - ndistinct + */ +static JsonParseErrorType +ndistinct_object_field_start(void *state, char *fname, bool isnull) +{ + NDistinctParseState *parse =3D state; + + if (strcmp(fname, PG_NDISTINCT_KEY_ATTRIBUTES) =3D=3D 0) + { + parse->found_attributes =3D true; + parse->state =3D NDIST_EXPECT_ATTNUM_LIST; + return JSON_SUCCESS; + } + + if (strcmp(fname, PG_NDISTINCT_KEY_NDISTINCT) =3D=3D 0) + { + parse->found_ndistinct =3D true; + parse->state =3D NDIST_EXPECT_NDISTINCT; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Only allowed keys are \"%s\" and \"%s\".", + PG_NDISTINCT_KEY_ATTRIBUTES, + PG_NDISTINCT_KEY_NDISTINCT)); + return JSON_SEM_ACTION_FAILED; +} + +/* + * Handle any array element. + */ +static JsonParseErrorType +ndistinct_array_element_start(void *state, bool isnull) +{ + NDistinctParseState *parse =3D state; + + switch (parse->state) + { + case NDIST_EXPECT_ATTNUM: + if (!isnull) + return JSON_SUCCESS; + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Attribute number array cannot be null.")); + break; + + case NDIST_EXPECT_ITEM: + if (!isnull) + return JSON_SUCCESS; + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Item list elements cannot be null.")); + + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Unexpected array element.")); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * Handle scalar events from the ndistinct input parser. + * + * Override integer parse error messages and replace them with errors + * specific to the context. + */ +static JsonParseErrorType +ndistinct_scalar(void *state, char *token, JsonTokenType tokentype) +{ + NDistinctParseState *parse =3D state; + AttrNumber attnum; + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + switch (parse->state) + { + case NDIST_EXPECT_ATTNUM: + attnum =3D pg_strtoint16_safe(token, (Node *) &escontext); + + if (!SOFT_ERROR_OCCURRED(&escontext)) + { + parse->attnum_list =3D lappend_int(parse->attnum_list, (int) attnum); + return JSON_SUCCESS; + } + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Invalid \"%s\" value.", PG_NDISTINCT_KEY_ATTRIBUTES)); + break; + + case NDIST_EXPECT_NDISTINCT: + + /* + * While the structure dictates that ndistinct is a double + * precision floating point, it has always been an integer in the + * output generated. Therefore, we parse it as an integer here. + */ + parse->ndistinct =3D pg_strtoint32_safe(token, (Node *) &escontext); + + if (!SOFT_ERROR_OCCURRED(&escontext)) + { + parse->state =3D NDIST_EXPECT_KEY; + return JSON_SUCCESS; + } + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Invalid \"%s\" value.", + PG_NDISTINCT_KEY_NDISTINCT)); + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", parse->str), + errdetail("Unexpected scalar.")); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * Compare the attribute arrays of two MVNDistinctItem values, + * looking for duplicate sets. + */ +static bool +has_duplicate_attributes(const MVNDistinctItem *a, + const MVNDistinctItem *b) +{ + if (a->nattributes !=3D b->nattributes) + return false; + + for (int i =3D 0; i < a->nattributes; i++) + { + if (a->attributes[i] !=3D b->attributes[i]) + return false; + } + + return true; +} + +/* + * Ensure that an attribute number appears as one of the attribute numbers + * in a MVNDistinctItem. + */ +static bool +item_has_attnum(const MVNDistinctItem *item, AttrNumber attnum) +{ + for (int i =3D 0; i < item->nattributes; i++) + { + if (attnum =3D=3D item->attributes[i]) + return true; + } + return false; +} + +/* + * Ensure that the attributes in MVNDistinctItem A are a subset of the + * reference MVNDistinctItem B. + */ +static bool +item_is_attnum_subset(const MVNDistinctItem *item, + const MVNDistinctItem *refitem) +{ + for (int i =3D 0; i < item->nattributes; i++) + { + if (!item_has_attnum(refitem, item->attributes[i])) + return false; + } + return true; +} + +/* + * Generate a string representing an array of attnum. + * + * Freeing the allocated string is the responsibility of the caller. + */ +static const char * +item_attnum_list(const MVNDistinctItem *item) +{ + StringInfoData str; + + initStringInfo(&str); + + appendStringInfo(&str, "%d", item->attributes[0]); + + for (int i =3D 1; i < item->nattributes; i++) + appendStringInfo(&str, ", %d", item->attributes[i]); + + return str.data; +} + +/* + * pg_ndistinct_in + * input routine for type pg_ndistinct. */ Datum pg_ndistinct_in(PG_FUNCTION_ARGS) { - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("cannot accept a value of type %s", "pg_ndistinct"))); + char *str =3D PG_GETARG_CSTRING(0); + NDistinctParseState parse_state; + JsonParseErrorType result; + JsonLexContext *lex; + JsonSemAction sem_action; + int item_most_attrs =3D 0; + int item_most_attrs_idx =3D 0; =20 - PG_RETURN_VOID(); /* keep compiler quiet */ + /* initialize semantic state */ + parse_state.str =3D str; + parse_state.state =3D NDIST_EXPECT_START; + parse_state.distinct_items =3D NIL; + parse_state.escontext =3D fcinfo->context; + parse_state.found_attributes =3D false; + parse_state.found_ndistinct =3D false; + parse_state.attnum_list =3D NIL; + parse_state.ndistinct =3D 0; + + /* set callbacks */ + sem_action.semstate =3D (void *) &parse_state; + sem_action.object_start =3D ndistinct_object_start; + sem_action.object_end =3D ndistinct_object_end; + sem_action.array_start =3D ndistinct_array_start; + sem_action.array_end =3D ndistinct_array_end; + sem_action.object_field_start =3D ndistinct_object_field_start; + sem_action.object_field_end =3D NULL; + sem_action.array_element_start =3D ndistinct_array_element_start; + sem_action.array_element_end =3D NULL; + sem_action.scalar =3D ndistinct_scalar; + + lex =3D makeJsonLexContextCstringLen(NULL, str, strlen(str), + PG_UTF8, true); + result =3D pg_parse_json(lex, &sem_action); + freeJsonLexContext(lex); + + if (result =3D=3D JSON_SUCCESS) + { + MVNDistinct *ndistinct; + int nitems =3D parse_state.distinct_items->length; + bytea *bytes; + + switch (parse_state.state) + { + case NDIST_EXPECT_COMPLETE: + + /* + * Parsing has ended correctly and we should have a list of + * items. If we don't, something has been done wrong in one + * of the earlier parsing steps. + */ + if (parse_state.distinct_items =3D=3D NIL) + elog(ERROR, + "cannot have empty item list after parsing success."); + break; + + case NDIST_EXPECT_START: + /* blank */ + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", str), + errdetail("Value cannot be empty.")); + PG_RETURN_NULL(); + break; + + default: + /* Unexpected end-state. */ + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", str), + errdetail("Unexpected end state %d.", parse_state.state)); + PG_RETURN_NULL(); + break; + } + + ndistinct =3D palloc(offsetof(MVNDistinct, items) + + nitems * sizeof(MVNDistinctItem)); + + ndistinct->magic =3D STATS_NDISTINCT_MAGIC; + ndistinct->type =3D STATS_NDISTINCT_TYPE_BASIC; + ndistinct->nitems =3D nitems; + + for (int i =3D 0; i < nitems; i++) + { + MVNDistinctItem *item =3D parse_state.distinct_items->elements[i].ptr_v= alue; + + /* + * Ensure that this item does not duplicate the attributes of any + * pre-existing item. + */ + for (int j =3D 0; j < i; j++) + { + if (has_duplicate_attributes(item, &ndistinct->items[j])) + { + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", str), + errdetail("Duplicated \"%s\" array found: [%s]", + PG_NDISTINCT_KEY_ATTRIBUTES, + item_attnum_list(item))); + PG_RETURN_NULL(); + } + } + + ndistinct->items[i].ndistinct =3D item->ndistinct; + ndistinct->items[i].nattributes =3D item->nattributes; + ndistinct->items[i].attributes =3D item->attributes; + + /* + * Keep track of the first longest attribute list. All other + * attribute lists must be a subset of this list. + */ + if (item->nattributes > item_most_attrs) + { + item_most_attrs =3D item->nattributes; + item_most_attrs_idx =3D i; + } + + /* + * Free the MVNDistinctItem, but not the attributes we're still + * using. + */ + pfree(item); + } + + /* + * Verify that all the sets of attribute numbers are a proper subset + * of the longest set recorded. This acts as an extra sanity check + * based on the input given. Note that this still needs to be + * cross-checked with the extended statistics objects this would be + * assigned to, but it provides one extra layer of protection. + */ + for (int i =3D 0; i < nitems; i++) + { + if (i =3D=3D item_most_attrs_idx) + continue; + + if (!item_is_attnum_subset(&ndistinct->items[i], + &ndistinct->items[item_most_attrs_idx])) + { + const MVNDistinctItem *item =3D &ndistinct->items[i]; + const MVNDistinctItem *refitem =3D &ndistinct->items[item_most_attrs_i= dx]; + const char *item_list =3D item_attnum_list(item); + const char *refitem_list =3D item_attnum_list(refitem); + + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", str), + errdetail("\"%s\" array: [%s] must be a subset of array: [%s]", + PG_NDISTINCT_KEY_ATTRIBUTES, + item_list, refitem_list)); + PG_RETURN_NULL(); + } + } + + bytes =3D statext_ndistinct_serialize(ndistinct); + + list_free(parse_state.distinct_items); + for (int i =3D 0; i < nitems; i++) + pfree(ndistinct->items[i].attributes); + pfree(ndistinct); + + PG_RETURN_BYTEA_P(bytes); + } + + /* + * If escontext already set, just use that. Anything else is a generic + * JSON parse error. + */ + if (!SOFT_ERROR_OCCURRED(parse_state.escontext)) + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_ndistinct: \"%s\"", str), + errdetail("Must be valid JSON.")); + + PG_RETURN_NULL(); } =20 /* diff --git a/src/test/regress/expected/pg_ndistinct.out b/src/test/regress/= expected/pg_ndistinct.out new file mode 100644 index 000000000000..705900585c76 --- /dev/null +++ b/src/test/regress/expected/pg_ndistinct.out @@ -0,0 +1,359 @@ +-- Tests for type pg_ndistinct +-- Invalid inputs +SELECT 'null'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "null" +LINE 1: SELECT 'null'::pg_ndistinct; + ^ +DETAIL: Unexpected scalar. +SELECT '{"a": 1}'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "{"a": 1}" +LINE 1: SELECT '{"a": 1}'::pg_ndistinct; + ^ +DETAIL: Initial element must be an array. +SELECT '[]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[]" +LINE 1: SELECT '[]'::pg_ndistinct; + ^ +DETAIL: Item array cannot be empty. +SELECT '{}'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "{}" +LINE 1: SELECT '{}'::pg_ndistinct; + ^ +DETAIL: Initial element must be an array. +SELECT '[null]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[null]" +LINE 1: SELECT '[null]'::pg_ndistinct; + ^ +DETAIL: Item list elements cannot be null. +SELECT * FROM pg_input_error_info('null', 'pg_ndistinct'); + message | detail | hint | sql_error_co= de=20 +--------------------------------+--------------------+------+-------------= --- + malformed pg_ndistinct: "null" | Unexpected scalar. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('{"a": 1}', 'pg_ndistinct'); + message | detail | = hint | sql_error_code=20 +------------------------------------+-----------------------------------+-= -----+---------------- + malformed pg_ndistinct: "{"a": 1}" | Initial element must be an array. | = | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[]', 'pg_ndistinct'); + message | detail | hint | sql_e= rror_code=20 +------------------------------+-----------------------------+------+------= ---------- + malformed pg_ndistinct: "[]" | Item array cannot be empty. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('{}', 'pg_ndistinct'); + message | detail | hint |= sql_error_code=20 +------------------------------+-----------------------------------+------+= ---------------- + malformed pg_ndistinct: "{}" | Initial element must be an array. | |= 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[null]', 'pg_ndistinct'); + message | detail | h= int | sql_error_code=20 +----------------------------------+------------------------------------+--= ----+---------------- + malformed pg_ndistinct: "[null]" | Item list elements cannot be null. | = | 22P02 +(1 row) + +-- Invalid keys +SELECT '[{"attributes_invalid" : [2,3], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes_invalid" : [2,3], "ndistinc= t" : 4}]" +LINE 1: SELECT '[{"attributes_invalid" : [2,3], "ndistinct" : 4}]'::... + ^ +DETAIL: Only allowed keys are "attributes" and "ndistinct". +SELECT '[{"attributes" : [2,3], "invalid" : 3, "ndistinct" : 4}]'::pg_ndis= tinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "invalid" : 3, "n= distinct" : 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "invalid" : 3, "ndistinct" :... + ^ +DETAIL: Only allowed keys are "attributes" and "ndistinct". +SELECT * FROM pg_input_error_info('[{"attributes_invalid" : [2,3], "ndisti= nct" : 4}]', 'pg_ndistinct'); + message = | detail | hint | sql_error= _code=20 +--------------------------------------------------------------------------= ---+-----------------------------------------------------+------+----------= ------ + malformed pg_ndistinct: "[{"attributes_invalid" : [2,3], "ndistinct" : 4}= ]" | Only allowed keys are "attributes" and "ndistinct". | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "invalid" : 3, = "ndistinct" : 4}]', 'pg_ndistinct'); + message = | detail | hint | sq= l_error_code=20 +--------------------------------------------------------------------------= ----------+-----------------------------------------------------+------+---= ------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "invalid" : 3, "ndistinc= t" : 4}]" | Only allowed keys are "attributes" and "ndistinct". | | 22= P02 +(1 row) + +-- Missing key +SELECT '[{"attributes" : [2,3]}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3]}]" +LINE 1: SELECT '[{"attributes" : [2,3]}]'::pg_ndistinct; + ^ +DETAIL: Item must contain "ndistinct" key. +SELECT '[{"ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"ndistinct" : 4}]" +LINE 1: SELECT '[{"ndistinct" : 4}]'::pg_ndistinct; + ^ +DETAIL: Item must contain "attributes" key. +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3]}]', 'pg_ndistinc= t'); + message | detail= | hint | sql_error_code=20 +----------------------------------------------------+---------------------= ---------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3]}]" | Item must contain "n= distinct" key. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"ndistinct" : 4}]', 'pg_ndistinct'); + message | detail = | hint | sql_error_code=20 +-----------------------------------------------+--------------------------= -----------+------+---------------- + malformed pg_ndistinct: "[{"ndistinct" : 4}]" | Item must contain "attrib= utes" key. | | 22P02 +(1 row) + +-- Special characters +SELECT '[{"\ud83d\ude04\ud83d\udc36" : [1, 2], "ndistinct" : 4}]'::pg_ndis= tinct; +ERROR: malformed pg_ndistinct: "[{"\ud83d\ude04\ud83d\udc36" : [1, 2], "n= distinct" : 4}]" +LINE 1: SELECT '[{"\ud83d\ude04\ud83d\udc36" : [1, 2], "ndistinct" :... + ^ +DETAIL: Only allowed keys are "attributes" and "ndistinct". +SELECT '[{"attributes" : [1, 2], "\ud83d\ude04\ud83d\udc36" : 4}]'::pg_ndi= stinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [1, 2], "\ud83d\ude04\ud= 83d\udc36" : 4}]" +LINE 1: SELECT '[{"attributes" : [1, 2], "\ud83d\ude04\ud83d\udc36" ... + ^ +DETAIL: Only allowed keys are "attributes" and "ndistinct". +SELECT '[{"attributes" : [1, 2], "ndistinct" : "\ud83d\ude04\ud83d\udc36"}= ]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [1, 2], "ndistinct" : "\= ud83d\ude04\ud83d\udc36"}]" +LINE 1: SELECT '[{"attributes" : [1, 2], "ndistinct" : "\ud83d\ude04... + ^ +DETAIL: Invalid "ndistinct" value. +SELECT '[{"attributes" : ["\ud83d\ude04\ud83d\udc36", 2], "ndistinct" : 1}= ]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : ["\ud83d\ude04\ud83d\udc= 36", 2], "ndistinct" : 1}]" +LINE 1: SELECT '[{"attributes" : ["\ud83d\ude04\ud83d\udc36", 2], "n... + ^ +DETAIL: Invalid "attributes" value. +-- Valid keys, invalid values +SELECT '[{"attributes" : null, "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : null, "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : null, "ndistinct" : 4}]'::pg_ndisti... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : [], "ndistinct" : 1}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [], "ndistinct" : 1}]" +LINE 1: SELECT '[{"attributes" : [], "ndistinct" : 1}]'::pg_ndistinc... + ^ +DETAIL: The "attributes" key must be an non-empty array. +SELECT '[{"attributes" : [2], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2], "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : [2], "ndistinct" : 4}]'::pg_ndistin... + ^ +DETAIL: The "ndistinct" key must contain an array of at least two attribu= tes. +SELECT '[{"attributes" : [2,null], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,null], "ndistinct" : = 4}]" +LINE 1: SELECT '[{"attributes" : [2,null], "ndistinct" : 4}]'::pg_nd... + ^ +DETAIL: Attribute number array cannot be null. +SELECT '[{"attributes" : [2,3], "ndistinct" : null}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : nul= l}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : null}]'::pg_nd... + ^ +DETAIL: Invalid "ndistinct" value. +SELECT '[{"attributes" : [2,"a"], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,"a"], "ndistinct" : 4= }]" +LINE 1: SELECT '[{"attributes" : [2,"a"], "ndistinct" : 4}]'::pg_ndi... + ^ +DETAIL: Invalid "attributes" value. +SELECT '[{"attributes" : [2,3], "ndistinct" : "a"}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : "a"= }]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : "a"}]'::pg_ndi... + ^ +DETAIL: Invalid "ndistinct" value. +SELECT '[{"attributes" : [2,3], "ndistinct" : []}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : []}= ]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : []}]'::pg_ndis... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : [2,3], "ndistinct" : [null]}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : [nu= ll]}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : [null]}]'::pg_... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : [2,3], "ndistinct" : [1,null]}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : [1,= null]}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : [1,null]}]'::p... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : [2,3], "ndistinct" : {"a": 1}}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : {"a= ": 1}}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : {"a": 1}}]'::p... + ^ +DETAIL: Value of "ndistinct" must be an integer. +SELECT '[{"attributes" : 1, "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : 1, "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : 1, "ndistinct" : 4}]'::pg_ndistinct... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : "a", "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : "a", "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : "a", "ndistinct" : 4}]'::pg_ndistin... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : {"a": 1}, "ndistinct" : 1}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : {"a": 1}, "ndistinct" : = 1}]" +LINE 1: SELECT '[{"attributes" : {"a": 1}, "ndistinct" : 1}]'::pg_nd... + ^ +DETAIL: Value of "attributes" must be an array of attribute numbers. +SELECT '[{"attributes" : [1, {"a": 1}], "ndistinct" : 1}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [1, {"a": 1}], "ndistinc= t" : 1}]" +LINE 1: SELECT '[{"attributes" : [1, {"a": 1}], "ndistinct" : 1}]'::... + ^ +DETAIL: Attribute lists can only contain attribute numbers. +SELECT * FROM pg_input_error_info('[{"attributes" : null, "ndistinct" : 4}= ]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +--------------------------------------------------------------------+-----= ---------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : null, "ndistinct" : 4}]" | Unex= pected scalar. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [], "ndistinct" : 1}]'= , 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +------------------------------------------------------------------+-------= -------------------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [], "ndistinct" : 1}]" | The "a= ttributes" key must be an non-empty array. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2], "ndistinct" : 4}]= ', 'pg_ndistinct'); + message | = detail | hint | s= ql_error_code=20 +-------------------------------------------------------------------+------= -----------------------------------------------------------------+------+--= -------------- + malformed pg_ndistinct: "[{"attributes" : [2], "ndistinct" : 4}]" | The "= ndistinct" key must contain an array of at least two attributes. | | 2= 2P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,null], "ndistinct" = : 4}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +------------------------------------------------------------------------+-= ---------------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,null], "ndistinct" : 4}]" | = Attribute number array cannot be null. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : n= ull}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +------------------------------------------------------------------------+-= ---------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : null}]" | = Invalid "ndistinct" value. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,"a"], "ndistinct" := 4}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +-----------------------------------------------------------------------+--= ---------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,"a"], "ndistinct" : 4}]" | I= nvalid "attributes" value. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : "= a"}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +-----------------------------------------------------------------------+--= --------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : "a"}]" | I= nvalid "ndistinct" value. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= ]}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +----------------------------------------------------------------------+---= -------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : []}]" | Ar= ray found in unexpected place. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= null]}]', 'pg_ndistinct'); + message = | detail | hint | sql_error_code=20 +--------------------------------------------------------------------------= +----------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : [null]}]" = | Array found in unexpected place. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= 1,null]}]', 'pg_ndistinct'); + message = | detail | hint | sql_error_code=20 +--------------------------------------------------------------------------= --+----------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : [1,null]}]= " | Array found in unexpected place. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : {= "a": 1}}]', 'pg_ndistinct'); + message = | detail | hint | sql_error_code=20 +--------------------------------------------------------------------------= --+------------------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : {"a": 1}}]= " | Value of "ndistinct" must be an integer. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : 1, "ndistinct" : 4}]',= 'pg_ndistinct'); + message | d= etail | hint | sql_error_code=20 +-----------------------------------------------------------------+--------= ------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : 1, "ndistinct" : 4}]" | Unexpec= ted scalar. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : "a", "ndistinct" : 4}]= ', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +-------------------------------------------------------------------+------= --------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : "a", "ndistinct" : 4}]" | Unexp= ected scalar. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : {"a": 1}, "ndistinct" = : 1}]', 'pg_ndistinct'); + message | = detail | hint | sql_e= rror_code=20 +------------------------------------------------------------------------+-= -------------------------------------------------------------+------+------= ---------- + malformed pg_ndistinct: "[{"attributes" : {"a": 1}, "ndistinct" : 1}]" | = Value of "attributes" must be an array of attribute numbers. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[{"attributes" : [1, {"a": 1}], "ndisti= nct" : 1}]', 'pg_ndistinct'); + message = | detail | hint | sql_error= _code=20 +--------------------------------------------------------------------------= ---+-----------------------------------------------------+------+----------= ------ + malformed pg_ndistinct: "[{"attributes" : [1, {"a": 1}], "ndistinct" : 1}= ]" | Attribute lists can only contain attribute numbers. | | 22P02 +(1 row) + +-- Duplicated attributes +SELECT '[{"attributes" : [2,2], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,2], "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : [2,2], "ndistinct" : 4}]'::pg_ndist... + ^ +DETAIL: Duplicated attribute number found: 2. +SELECT * FROM pg_input_error_info('[{"attributes" : [2,2], "ndistinct" : 4= }]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +---------------------------------------------------------------------+----= -----------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,2], "ndistinct" : 4}]" | Dup= licated attribute number found: 2. | | 22P02 +(1 row) + +-- Duplicated attribute lists. +SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,3], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,3], "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + ^ +DETAIL: Duplicated "attributes" array found: [2, 3] +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : 4= }, + {"attributes" : [2,3], "ndistinct" : 4}]', 'pg_ndistinct'); + message | = detail | hint | sql_error_code=20 +--------------------------------------------------------------------+-----= ----------------------------------------+------+---------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : 4},+| Dupl= icated "attributes" array found: [2, 3] | | 22P02 + {"attributes" : [2,3], "ndistinct" : 4}]" | = | |=20 +(1 row) + +-- Partially-covered attribute lists. +SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]'::pg_ndistinct; +ERROR: malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + ^ +DETAIL: "attributes" array: [2, 3] must be a subset of array: [1, 3, -1, = -2] +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : 4= }, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]', 'pg_ndistinct'); + message | = detail | hint | s= ql_error_code=20 +--------------------------------------------------------------------+-----= -----------------------------------------------------------------+------+--= -------------- + malformed pg_ndistinct: "[{"attributes" : [2,3], "ndistinct" : 4},+| "att= ributes" array: [2, 3] must be a subset of array: [1, 3, -1, -2] | | 2= 2P02 + {"attributes" : [2,-1], "ndistinct" : 4}, +| = | |= =20 + {"attributes" : [2,3,-1], "ndistinct" : 4}, +| = | |= =20 + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]" | = | |= =20 +(1 row) + +-- Valid inputs +-- Two attributes. +SELECT '[{"attributes" : [1,2], "ndistinct" : 4}]'::pg_ndistinct; + pg_ndistinct =20 +------------------------------------------ + [{"attributes": [1, 2], "ndistinct": 4}] +(1 row) + +-- Three attributes. +SELECT '[{"attributes" : [-1,2], "ndistinct" : 1}, + {"attributes" : [-1,3], "ndistinct" : 2}, + {"attributes" : [-1,2,3], "ndistinct" : 3}]'::pg_ndistinct; + pg_ndistinct = =20 +--------------------------------------------------------------------------= ------------------------------------------------------ + [{"attributes": [-1, 2], "ndistinct": 1}, {"attributes": [-1, 3], "ndisti= nct": 2}, {"attributes": [-1, 2, 3], "ndistinct": 3}] +(1 row) + diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel= _schedule index f56482fb9f12..f3f0b5f2f317 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -28,7 +28,7 @@ test: strings md5 numerology point lseg line box path pol= ygon circle date time t # geometry depends on point, lseg, line, box, path, polygon, circle # horology depends on date, time, timetz, timestamp, timestamptz, interval # ---------- -test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity c= omments expressions unicode xid mvcc database stats_import +test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity c= omments expressions unicode xid mvcc database stats_import pg_ndistinct =20 # ---------- # Load huge amounts of data diff --git a/src/test/regress/sql/pg_ndistinct.sql b/src/test/regress/sql/p= g_ndistinct.sql new file mode 100644 index 000000000000..dde508a31f50 --- /dev/null +++ b/src/test/regress/sql/pg_ndistinct.sql @@ -0,0 +1,87 @@ +-- Tests for type pg_ndistinct + +-- Invalid inputs +SELECT 'null'::pg_ndistinct; +SELECT '{"a": 1}'::pg_ndistinct; +SELECT '[]'::pg_ndistinct; +SELECT '{}'::pg_ndistinct; +SELECT '[null]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('null', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('{"a": 1}', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('{}', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[null]', 'pg_ndistinct'); +-- Invalid keys +SELECT '[{"attributes_invalid" : [2,3], "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "invalid" : 3, "ndistinct" : 4}]'::pg_ndis= tinct; +SELECT * FROM pg_input_error_info('[{"attributes_invalid" : [2,3], "ndisti= nct" : 4}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "invalid" : 3, = "ndistinct" : 4}]', 'pg_ndistinct'); + +-- Missing key +SELECT '[{"attributes" : [2,3]}]'::pg_ndistinct; +SELECT '[{"ndistinct" : 4}]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3]}]', 'pg_ndistinc= t'); +SELECT * FROM pg_input_error_info('[{"ndistinct" : 4}]', 'pg_ndistinct'); + +-- Special characters +SELECT '[{"\ud83d\ude04\ud83d\udc36" : [1, 2], "ndistinct" : 4}]'::pg_ndis= tinct; +SELECT '[{"attributes" : [1, 2], "\ud83d\ude04\ud83d\udc36" : 4}]'::pg_ndi= stinct; +SELECT '[{"attributes" : [1, 2], "ndistinct" : "\ud83d\ude04\ud83d\udc36"}= ]'::pg_ndistinct; +SELECT '[{"attributes" : ["\ud83d\ude04\ud83d\udc36", 2], "ndistinct" : 1}= ]'::pg_ndistinct; + +-- Valid keys, invalid values +SELECT '[{"attributes" : null, "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : [], "ndistinct" : 1}]'::pg_ndistinct; +SELECT '[{"attributes" : [2], "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,null], "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : null}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,"a"], "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : "a"}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : []}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : [null]}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : [1,null]}]'::pg_ndistinct; +SELECT '[{"attributes" : [2,3], "ndistinct" : {"a": 1}}]'::pg_ndistinct; +SELECT '[{"attributes" : 1, "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : "a", "ndistinct" : 4}]'::pg_ndistinct; +SELECT '[{"attributes" : {"a": 1}, "ndistinct" : 1}]'::pg_ndistinct; +SELECT '[{"attributes" : [1, {"a": 1}], "ndistinct" : 1}]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('[{"attributes" : null, "ndistinct" : 4}= ]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [], "ndistinct" : 1}]'= , 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2], "ndistinct" : 4}]= ', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,null], "ndistinct" = : 4}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : n= ull}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,"a"], "ndistinct" := 4}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : "= a"}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= ]}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= null]}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : [= 1,null]}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : {= "a": 1}}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : 1, "ndistinct" : 4}]',= 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : "a", "ndistinct" : 4}]= ', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : {"a": 1}, "ndistinct" = : 1}]', 'pg_ndistinct'); +SELECT * FROM pg_input_error_info('[{"attributes" : [1, {"a": 1}], "ndisti= nct" : 1}]', 'pg_ndistinct'); +-- Duplicated attributes +SELECT '[{"attributes" : [2,2], "ndistinct" : 4}]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('[{"attributes" : [2,2], "ndistinct" : 4= }]', 'pg_ndistinct'); +-- Duplicated attribute lists. +SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,3], "ndistinct" : 4}]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : 4= }, + {"attributes" : [2,3], "ndistinct" : 4}]', 'pg_ndistinct'); +-- Partially-covered attribute lists. +SELECT '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]'::pg_ndistinct; +SELECT * FROM pg_input_error_info('[{"attributes" : [2,3], "ndistinct" : 4= }, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [1,3,-1,-2], "ndistinct" : 4}]', 'pg_ndistinct'); + +-- Valid inputs +-- Two attributes. +SELECT '[{"attributes" : [1,2], "ndistinct" : 4}]'::pg_ndistinct; +-- Three attributes. +SELECT '[{"attributes" : [-1,2], "ndistinct" : 1}, + {"attributes" : [-1,3], "ndistinct" : 2}, + {"attributes" : [-1,2,3], "ndistinct" : 3}]'::pg_ndistinct; diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs= =2Elist index 23bce72ae64b..03a8d74114d0 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -1729,6 +1729,8 @@ MultirangeIOData MultirangeParseState MultirangeType NDBOX +NDistinctParseState +NDistinctSemanticState NLSVERSIONINFOEX NODE NTSTATUS --=20 2.51.0 --AorqrXjCfzfo1Gz1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v14-0002-Add-working-input-function-for-pg_dependencies.patch" Content-Transfer-Encoding: quoted-printable =46rom 9f60d150fc8449601edfaf9f9fcdfb041963b607 Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Mon, 17 Nov 2025 15:49:36 +0900 Subject: [PATCH v14 2/5] Add working input function for pg_dependencies. This will consume the format that was established when the output function for pg_dependencies was recently changed. This will be needed for importing extended statistics. --- src/backend/utils/adt/pg_dependencies.c | 742 +++++++++++++++++- src/backend/utils/adt/pg_ndistinct.c | 3 +- src/test/regress/expected/pg_dependencies.out | 176 +++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/pg_dependencies.sql | 47 ++ 5 files changed, 957 insertions(+), 13 deletions(-) create mode 100644 src/test/regress/expected/pg_dependencies.out create mode 100644 src/test/regress/sql/pg_dependencies.sql diff --git a/src/backend/utils/adt/pg_dependencies.c b/src/backend/utils/ad= t/pg_dependencies.c index 87181aa00e9a..723e322a491f 100644 --- a/src/backend/utils/adt/pg_dependencies.c +++ b/src/backend/utils/adt/pg_dependencies.c @@ -14,29 +14,751 @@ =20 #include "postgres.h" =20 +#include "common/int.h" +#include "common/jsonapi.h" #include "lib/stringinfo.h" +#include "mb/pg_wchar.h" +#include "nodes/miscnodes.h" #include "statistics/extended_stats_internal.h" #include "statistics/statistics_format.h" +#include "utils/builtins.h" +#include "utils/float.h" #include "utils/fmgrprotos.h" =20 +typedef enum +{ + DEPS_EXPECT_START =3D 0, + DEPS_EXPECT_ITEM, + DEPS_EXPECT_KEY, + DEPS_EXPECT_ATTNUM_LIST, + DEPS_EXPECT_ATTNUM, + DEPS_EXPECT_DEPENDENCY, + DEPS_EXPECT_DEGREE, + DEPS_PARSE_COMPLETE +} DepsParseSemanticState; + +typedef struct +{ + const char *str; + DepsParseSemanticState state; + + List *dependency_list; + Node *escontext; + + bool found_attributes; /* Item has an attributes key */ + bool found_dependency; /* Item has an dependency key */ + bool found_degree; /* Item has degree key */ + List *attnum_list; /* Accumulated attributes attnums */ + AttrNumber dependency; + double degree; +} DependenciesParseState; + +/* + * Invoked at the start of each MVDependency object. + * + * The entire JSON document should be one array of MVDependency objects. + * + * If we are anywhere else in the document, it's an error. + */ +static JsonParseErrorType +dependencies_object_start(void *state) +{ + DependenciesParseState *parse =3D state; + + switch(parse->state) + { + case DEPS_EXPECT_ITEM: + /* Now we expect to see attributes/dependency/degree keys */ + parse->state =3D DEPS_EXPECT_KEY; + return JSON_SUCCESS; + break; + + case DEPS_EXPECT_START: + /* pg_dependencies must begin with a '[' */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Initial element must be an array.")); + break; + + case DEPS_EXPECT_KEY: + /* In an object, expecting key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Expected an object key.")); + break; + + case DEPS_EXPECT_ATTNUM_LIST: + /* Just followed an "attributes": key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Value of \"%s\" must be an array of attribute numbers.", + PG_DEPENDENCIES_KEY_ATTRIBUTES)); + break; + + case DEPS_EXPECT_ATTNUM: + /* In an attnum list, expect only scalar integers */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Attribute lists can only contain attribute numbers.")); + break; + + case DEPS_EXPECT_DEPENDENCY: + /* Just followed a "dependency" key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Value of \"%s\" must be an integer.", + PG_DEPENDENCIES_KEY_DEPENDENCY)); + break; + + case DEPS_EXPECT_DEGREE: + /* Just followed a "degree" key */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Value of \"%s\" must be an integer.", + PG_DEPENDENCIES_KEY_DEGREE)); + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Unexpected parse state: %d", (int) parse->state)); + } + + return JSON_SEM_ACTION_FAILED; +} + +static int +attnum_compare(const void *aptr, const void *bptr) +{ + AttrNumber a =3D *(const AttrNumber *) aptr; + AttrNumber b =3D *(const AttrNumber *) bptr; + + return pg_cmp_s16(a, b); +} + +static JsonParseErrorType +dependencies_object_end(void *state) +{ + DependenciesParseState *parse =3D state; + + MVDependency *dep; + AttrNumber *attrsort; + + int natts =3D 0; + + if (parse->state !=3D DEPS_EXPECT_KEY) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Unexpected parse state: %d", (int) parse->state)); + return JSON_SEM_ACTION_FAILED; + } + + if (!parse->found_attributes) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Item must contain \"%s\" key", + PG_DEPENDENCIES_KEY_ATTRIBUTES)); + return JSON_SEM_ACTION_FAILED; + } + + if (!parse->found_dependency) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Item must contain \"%s\" key.", + PG_DEPENDENCIES_KEY_DEPENDENCY)); + return JSON_SEM_ACTION_FAILED; + } + + if (!parse->found_degree) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Item must contain \"%s\" key.", + PG_DEPENDENCIES_KEY_DEGREE)); + return JSON_SEM_ACTION_FAILED; + } + + /* + * We need at least one attribute number a dependencies item, anything + * less is malformed. + */ + natts =3D parse->attnum_list->length; + if (natts < 1) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("The \"%s\" key must contain an array of at least one elemen= t.", + PG_DEPENDENCIES_KEY_ATTRIBUTES)); + return JSON_SEM_ACTION_FAILED; + } + attrsort =3D palloc0(natts * sizeof(AttrNumber)); + + /* + * Allocate enough space for the dependency, the attnums in the list, plus + * the final attnum + */ + dep =3D palloc0(offsetof(MVDependency, attributes) + ((natts + 1) * sizeo= f(AttrNumber))); + dep->nattributes =3D natts + 1; + + dep->attributes[natts] =3D parse->dependency; + dep->degree =3D parse->degree; + + attrsort =3D palloc0(dep->nattributes * sizeof(AttrNumber)); + attrsort[natts] =3D parse->dependency; + + for (int i =3D 0; i < natts; i++) + { + attrsort[i] =3D (AttrNumber) parse->attnum_list->elements[i].int_value; + dep->attributes[i] =3D attrsort[i]; + } + + /* Check attrsort for uniqueness */ + qsort(attrsort, natts + 1, sizeof(AttrNumber), attnum_compare); + for (int i =3D 1; i < dep->nattributes; i++) + { + if (attrsort[i] =3D=3D attrsort[i - 1]) + { + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Duplicated attribute number found: %d.", attrsort[i])); + pfree(attrsort); + pfree(dep); + return JSON_SEM_ACTION_FAILED; + } + } + pfree(attrsort); + + parse->dependency_list =3D lappend(parse->dependency_list, (void *) dep); + + /* Reset dependency item state variables */ + list_free(parse->attnum_list); + parse->attnum_list =3D NIL; + parse->dependency =3D 0; + parse->degree =3D 0.0; + parse->found_attributes =3D false; + parse->found_dependency =3D false; + parse->found_degree =3D false; + + /* Now we are looking for the next MVDependency */ + parse->state =3D DEPS_EXPECT_ITEM; + return JSON_SUCCESS; +} + +/* + * Dependency input format does not have arrays, so any array elements + * encountered are an error. + */ +static JsonParseErrorType +dependencies_array_start(void *state) +{ + DependenciesParseState *parse =3D state; + + switch (parse->state) + { + case DEPS_EXPECT_ATTNUM_LIST: + parse->state =3D DEPS_EXPECT_ATTNUM; + break; + case DEPS_EXPECT_START: + parse->state =3D DEPS_EXPECT_ITEM; + break; + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Array found in unexpected place.")); + return JSON_SEM_ACTION_FAILED; + } + + return JSON_SUCCESS; +} + +/* + * Either the end of an attnum list or the whole object. + */ +static JsonParseErrorType +dependencies_array_end(void *state) +{ + DependenciesParseState *parse =3D state; + + switch (parse->state) + { + case DEPS_EXPECT_ATTNUM: + if (parse->attnum_list !=3D NIL) + { + parse->state =3D DEPS_EXPECT_KEY; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("The \"%s\" key must be an non-empty array.", + PG_DEPENDENCIES_KEY_ATTRIBUTES)); + break; + + case DEPS_EXPECT_ITEM: + if (parse->dependency_list !=3D NIL) + { + parse->state =3D DEPS_PARSE_COMPLETE; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Item array cannot be empty.")); + break; + + default: + /* + * This can only happen if a case was missed in depenenceies_array_star= t() + */ + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Array found in unexpected place.")); + } + return JSON_SEM_ACTION_FAILED; +} + +/* + * The valid keys for the MVDependency object are: + * - attributes + * - depeendency + * - degree + */ +static JsonParseErrorType +dependencies_object_field_start(void *state, char *fname, bool isnull) +{ + DependenciesParseState *parse =3D state; + + if (strcmp(fname, PG_DEPENDENCIES_KEY_ATTRIBUTES) =3D=3D 0) + { + parse->found_attributes =3D true; + parse->state =3D DEPS_EXPECT_ATTNUM_LIST; + return JSON_SUCCESS; + } + + if (strcmp(fname, PG_DEPENDENCIES_KEY_DEPENDENCY) =3D=3D 0) + { + parse->found_dependency =3D true; + parse->state =3D DEPS_EXPECT_DEPENDENCY; + return JSON_SUCCESS; + } + + if (strcmp(fname, PG_DEPENDENCIES_KEY_DEGREE) =3D=3D 0) + { + parse->found_degree =3D true; + parse->state =3D DEPS_EXPECT_DEGREE; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Only allowed keys are \"%s\", \"%s\" and \"%s\".", + PG_DEPENDENCIES_KEY_ATTRIBUTES, + PG_DEPENDENCIES_KEY_DEPENDENCY, + PG_DEPENDENCIES_KEY_DEGREE)); + return JSON_SEM_ACTION_FAILED; +} + +/* + * pg_dependencies input format does not have arrays, so any array elements + * encountered are an error. + */ +static JsonParseErrorType +dependencies_array_element_start(void *state, bool isnull) +{ + DependenciesParseState *parse =3D state; + + switch(parse->state) + { + case DEPS_EXPECT_ATTNUM: + if (!isnull) + return JSON_SUCCESS; + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Attribute number array cannot be null.")); + + return JSON_SEM_ACTION_FAILED; + break; + + case DEPS_EXPECT_ITEM: + if (!isnull) + return JSON_SUCCESS; + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Item list elements cannot be null.")); + + return JSON_SEM_ACTION_FAILED; + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Unexpected array element.")); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * Handle scalar events from the dependencies input parser. + * + * There is only one case where we will encounter a scalar, and that is the + * dependency degree for the previous object key. + */ +static JsonParseErrorType +dependencies_scalar(void *state, char *token, JsonTokenType tokentype) +{ + DependenciesParseState *parse =3D state; + AttrNumber attnum; + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + switch(parse->state) + { + case DEPS_EXPECT_ATTNUM: + attnum =3D pg_strtoint16_safe(token, (Node *) &escontext); + + if (!SOFT_ERROR_OCCURRED(&escontext)) + { + parse->attnum_list =3D lappend_int(parse->attnum_list, (int) attnum); + /* No state change, we expect more attnums */ + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Invalid \"%s\" value.", PG_DEPENDENCIES_KEY_ATTRIBUTES)); + break; + + case DEPS_EXPECT_DEPENDENCY: + parse->dependency =3D (AttrNumber) + pg_strtoint16_safe(token, (Node *) &escontext); + + if (!SOFT_ERROR_OCCURRED(&escontext)) + { + parse->state =3D DEPS_EXPECT_KEY; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Invalid \"%s\" value.", PG_DEPENDENCIES_KEY_DEPENDENCY)); + break; + + case DEPS_EXPECT_DEGREE: + parse->degree =3D float8in_internal(token, NULL, "double", + token, (Node *) &escontext); + + if (!SOFT_ERROR_OCCURRED(&escontext)) + { + parse->state =3D DEPS_EXPECT_KEY; + return JSON_SUCCESS; + } + + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Invalid \"%s\" value.", PG_DEPENDENCIES_KEY_DEGREE)); + break; + + default: + errsave(parse->escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", parse->str), + errdetail("Unexpected scalar.")); + } + + return JSON_SEM_ACTION_FAILED; +} + +/* + * Compare the attribute arrays of two MVDependency values, + * looking for duplicate sets. + */ +static bool +has_duplicate_attributes(const MVDependency *a, const MVDependency *b) +{ + int i; + + if (a->nattributes !=3D b->nattributes) + return false; + + for (i =3D 0; i < a->nattributes; i++) + { + if (a->attributes[i] !=3D b->attributes[i]) + return false; + } + + return true; +} + +/* + * Ensure that an attnum appears as one of the attnums in a given + * MVDependency. + */ +static bool +dep_has_attnum(const MVDependency *item, AttrNumber attnum) +{ + for (int i =3D 0; i < item->nattributes; i++) + { + if (attnum =3D=3D item->attributes[i]) + return true; + } + return false; +} + +/* + * Ensure that the attributes of one MVDependency A are a proper subset + * of the reference MVDependency B. + */ +static bool +dep_is_attnum_subset(const MVDependency *item, + const MVDependency *refitem) +{ + for (int i =3D 0; i < item->nattributes; i++) + { + if (!dep_has_attnum(refitem,item->attributes[i])) + return false; + } + return true; +} + +/* + * Generate a string representing an array of attnums. Internally, the + * dependency attribute is the last element, so we leave that off. + * + * + * Freeing the allocated string is responsibility of the caller. + */ +static const char * +dep_attnum_list(const MVDependency *item) +{ + StringInfoData str; + + initStringInfo(&str); + + appendStringInfo(&str, "%d", item->attributes[0]); + + for (int i =3D 1; i < item->nattributes - 1; i++) + appendStringInfo(&str, ", %d", item->attributes[i]); + + return str.data; +} + +/* + * Return the dependency, which is the last attribute element. + */ +static const AttrNumber +dep_attnum_dependency(const MVDependency *item) +{ + return item->attributes[item->nattributes - 1]; +} + /* * pg_dependencies_in - input routine for type pg_dependencies. * - * pg_dependencies is real enough to be a table column, but it has no oper= ations - * of its own, and disallows input too + * This format is valid JSON, with the expected format: + * [{"attributes": [1,2], "dependency": -1, "degree": 1.0000}, + * {"attributes": [1,-1], "dependency": 2, "degree": 0.0000}, + * {"attributes": [2,-1], "dependency": 1, "degree": 1.0000}] + * */ Datum pg_dependencies_in(PG_FUNCTION_ARGS) { - /* - * pg_node_list stores the data in binary form and parsing text input is - * not needed, so disallow this. - */ - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("cannot accept a value of type %s", "pg_dependencies"))); + char *str =3D PG_GETARG_CSTRING(0); =20 - PG_RETURN_VOID(); /* keep compiler quiet */ + DependenciesParseState parse_state; + JsonParseErrorType result; + JsonLexContext *lex; + JsonSemAction sem_action; + + /* initialize the semantic state */ + parse_state.str =3D str; + parse_state.state =3D DEPS_EXPECT_START; + parse_state.dependency_list =3D NIL; + parse_state.attnum_list =3D NIL; + parse_state.dependency =3D 0; + parse_state.degree =3D 0.0; + parse_state.found_attributes =3D false; + parse_state.found_dependency =3D false; + parse_state.found_degree =3D false; + parse_state.escontext =3D fcinfo->context; + + /* set callbacks */ + sem_action.semstate =3D (void *) &parse_state; + sem_action.object_start =3D dependencies_object_start; + sem_action.object_end =3D dependencies_object_end; + sem_action.array_start =3D dependencies_array_start; + sem_action.array_end =3D dependencies_array_end; + sem_action.array_element_start =3D dependencies_array_element_start; + sem_action.array_element_end =3D NULL; + sem_action.object_field_start =3D dependencies_object_field_start; + sem_action.object_field_end =3D NULL; + sem_action.scalar =3D dependencies_scalar; + + lex =3D makeJsonLexContextCstringLen(NULL, str, strlen(str), PG_UTF8, tru= e); + + result =3D pg_parse_json(lex, &sem_action); + freeJsonLexContext(lex); + + if (result =3D=3D JSON_SUCCESS) + { + List *list =3D parse_state.dependency_list; + int ndeps =3D list->length; + MVDependencies *mvdeps; + bytea *bytes; + + int dep_most_attrs =3D 0; + int dep_most_attrs_idx =3D 0; + + switch(parse_state.state) + { + case DEPS_PARSE_COMPLETE: + /* + * Parse ended in the expected place. We should have a list of items, + * but if we don't it is because there are bugs in other parse steps. + */ + if (parse_state.dependency_list =3D=3D NIL) + elog(ERROR, + "pg_dependencies parssing claims success with an empty item list."); + + break; + + case DEPS_EXPECT_START: + /* blank */ + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", str), + errdetail("Value cannot be empty.")); + PG_RETURN_NULL(); + break; + + default: + /* Unexpected end-state. TODO: Is this an elog()? */ + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", str), + errdetail("Unexpected end state %d.", parse_state.state)); + PG_RETURN_NULL(); + break; + } + + mvdeps =3D palloc0(offsetof(MVDependencies, deps) + ndeps * sizeof(MVDep= endency)); + mvdeps->magic =3D STATS_DEPS_MAGIC; + mvdeps->type =3D STATS_DEPS_TYPE_BASIC; + mvdeps->ndeps =3D ndeps; + + /* copy MVDependency structs out of the list into the MVDependencies */ + for (int i =3D 0; i < ndeps; i++) + { + mvdeps->deps[i] =3D list->elements[i].ptr_value; + + /* + * Ensure that this item does not duplicate the attributes of any + * pre-existing item. + */ + for (int j =3D 0; j < i; j++) + { + if (has_duplicate_attributes(mvdeps->deps[i], mvdeps->deps[j])) + { + MVDependency *dep =3D mvdeps->deps[i]; + + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", str), + errdetail("Duplicate \"" PG_DEPENDENCIES_KEY_ATTRIBUTES "\" array: = [%s]" + " with \"" PG_DEPENDENCIES_KEY_DEPENDENCY "\": %d.", + dep_attnum_list(dep), dep_attnum_dependency(dep))); + PG_RETURN_NULL(); + } + } + + /* + * Keep track of the first longest attribute list. All other attribute + * lists must be a subset of this list. + */ + if (mvdeps->deps[i]->nattributes > dep_most_attrs) + { + dep_most_attrs =3D mvdeps->deps[i]->nattributes; + dep_most_attrs_idx =3D i; + } + } + + /* + * Verify that all attnum sets are a proper subset of the first longest + * attnum set. + */ + for (int i =3D 0; i < ndeps; i++) + { + if (i =3D=3D dep_most_attrs_idx) + continue; + + if (!dep_is_attnum_subset(mvdeps->deps[i], + mvdeps->deps[dep_most_attrs_idx])) + { + MVDependency *dep =3D mvdeps->deps[i]; + MVDependency *refdep =3D mvdeps->deps[dep_most_attrs_idx]; + const char *dep_list =3D dep_attnum_list(dep); + const char *refdep_list =3D dep_attnum_list(refdep); + + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", str), + errdetail("\"" PG_DEPENDENCIES_KEY_ATTRIBUTES "\" array: [%s]" + " with dependency %d must be a subset of array: [%s]" + " with dependency %d.", + dep_list, dep_attnum_dependency(dep), + refdep_list, dep_attnum_dependency(refdep))); + PG_RETURN_NULL(); + } + } + bytes =3D statext_dependencies_serialize(mvdeps); + + list_free(list); + for (int i =3D 0; i < ndeps; i++) + pfree(mvdeps->deps[i]); + pfree(mvdeps); + + PG_RETURN_BYTEA_P(bytes); + } + + /* + * If escontext already set, just use that. + * Anything else is a generic JSON parse error. + */ + if (!SOFT_ERROR_OCCURRED(parse_state.escontext)) + errsave(parse_state.escontext, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("malformed pg_dependencies: \"%s\"", str), + errdetail("Must be valid JSON.")); + + PG_RETURN_NULL(); /* keep compiler quiet */ } =20 /* diff --git a/src/backend/utils/adt/pg_ndistinct.c b/src/backend/utils/adt/p= g_ndistinct.c index 4c13b2e62885..9f83f3373527 100644 --- a/src/backend/utils/adt/pg_ndistinct.c +++ b/src/backend/utils/adt/pg_ndistinct.c @@ -447,8 +447,7 @@ ndistinct_scalar(void *state, char *token, JsonTokenTyp= e tokentype) errsave(parse->escontext, errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("malformed pg_ndistinct: \"%s\"", parse->str), - errdetail("Invalid \"%s\" value.", - PG_NDISTINCT_KEY_NDISTINCT)); + errdetail("Invalid \"%s\" value.", PG_NDISTINCT_KEY_NDISTINCT)); break; =20 default: diff --git a/src/test/regress/expected/pg_dependencies.out b/src/test/regre= ss/expected/pg_dependencies.out new file mode 100644 index 000000000000..120623717999 --- /dev/null +++ b/src/test/regress/expected/pg_dependencies.out @@ -0,0 +1,176 @@ +-- Tests for type pg_distinct +-- Invalid inputs +SELECT 'null'::pg_dependencies; +ERROR: malformed pg_dependencies: "null" +LINE 1: SELECT 'null'::pg_dependencies; + ^ +DETAIL: Unexpected scalar. +SELECT '{"a": 1}'::pg_dependencies; +ERROR: malformed pg_dependencies: "{"a": 1}" +LINE 1: SELECT '{"a": 1}'::pg_dependencies; + ^ +DETAIL: Initial element must be an array. +SELECT '[]'::pg_dependencies; +ERROR: malformed pg_dependencies: "[]" +LINE 1: SELECT '[]'::pg_dependencies; + ^ +DETAIL: Item array cannot be empty. +SELECT '{}'::pg_dependencies; +ERROR: malformed pg_dependencies: "{}" +LINE 1: SELECT '{}'::pg_dependencies; + ^ +DETAIL: Initial element must be an array. +SELECT '[null]'::pg_dependencies; +ERROR: malformed pg_dependencies: "[null]" +LINE 1: SELECT '[null]'::pg_dependencies; + ^ +DETAIL: Item list elements cannot be null. +SELECT * FROM pg_input_error_info('null', 'pg_dependencies'); + message | detail | hint | sql_error= _code=20 +-----------------------------------+--------------------+------+----------= ------ + malformed pg_dependencies: "null" | Unexpected scalar. | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('{"a": 1}', 'pg_dependencies'); + message | detail = | hint | sql_error_code=20 +---------------------------------------+----------------------------------= -+------+---------------- + malformed pg_dependencies: "{"a": 1}" | Initial element must be an array.= | | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[]', 'pg_dependencies'); + message | detail | hint | sq= l_error_code=20 +---------------------------------+-----------------------------+------+---= ------------- + malformed pg_dependencies: "[]" | Item array cannot be empty. | | 22= P02 +(1 row) + +SELECT * FROM pg_input_error_info('{}', 'pg_dependencies'); + message | detail | hin= t | sql_error_code=20 +---------------------------------+-----------------------------------+----= --+---------------- + malformed pg_dependencies: "{}" | Initial element must be an array. | = | 22P02 +(1 row) + +SELECT * FROM pg_input_error_info('[null]', 'pg_dependencies'); + message | detail = | hint | sql_error_code=20 +-------------------------------------+------------------------------------= +------+---------------- + malformed pg_dependencies: "[null]" | Item list elements cannot be null. = | | 22P02 +(1 row) + +-- Invalid keys +SELECT '[{"attributes_invalid" : [2,3], "dependency" : 4}]'::pg_dependenci= es; +ERROR: malformed pg_dependencies: "[{"attributes_invalid" : [2,3], "depen= dency" : 4}]" +LINE 1: SELECT '[{"attributes_invalid" : [2,3], "dependency" : 4}]':... + ^ +DETAIL: Only allowed keys are "attributes", "dependency" and "degree". +SELECT '[{"attributes" : [2,3], "invalid" : 3, "dependency" : 4}]'::pg_dep= endencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "invalid" : 3,= "dependency" : 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "invalid" : 3, "dependency" ... + ^ +DETAIL: Only allowed keys are "attributes", "dependency" and "degree". +-- Missing keys +SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_depe... + ^ +DETAIL: Item must contain "degree" key. +SELECT '[{"attributes" : [2,3], "degree" : 1.000}]'::pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "degree" : 1.0= 00}]" +LINE 1: SELECT '[{"attributes" : [2,3], "degree" : 1.000}]'::pg_depe... + ^ +DETAIL: Item must contain "dependency" key. +SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := 4}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_depe... + ^ +DETAIL: Item must contain "degree" key. +-- Valid keys, invalid values +SELECT '[{"attributes" : null, "dependency" : 4, "degree": 1.000}]'::pg_de= pendencies; +ERROR: malformed pg_dependencies: "[{"attributes" : null, "dependency" : = 4, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : null, "dependency" : 4, "degree": 1... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : [2,null], "dependency" : 4, "degree": 1.000}]'::p= g_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,null], "dependency= " : 4, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,null], "dependency" : 4, "degree... + ^ +DETAIL: Attribute number array cannot be null. +SELECT '[{"attributes" : [2,3], "dependency" : null, "degree": 1.000}]'::p= g_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := null, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : null, "degree... + ^ +DETAIL: Invalid "dependency" value. +SELECT '[{"attributes" : [2,"a"], "dependency" : 4, "degree": 1.000}]'::pg= _dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,"a"], "dependency"= : 4, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,"a"], "dependency" : 4, "degree"... + ^ +DETAIL: Invalid "attributes" value. +SELECT '[{"attributes" : [2,3], "dependency" : "a", "degree": 1.000}]'::pg= _dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := "a", "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : "a", "degree"... + ^ +DETAIL: Invalid "dependency" value. +SELECT '[{"attributes" : [2,3], "dependency" : [], "degree": 1.000}]'::pg_= dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := [], "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : [], "degree":... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : [2,3], "dependency" : [null], "degree": 1.000}]':= :pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := [null], "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : [null], "degr... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : [2,3], "dependency" : [1,null], "degree": 1.000}]= '::pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := [1,null], "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : [1,null], "de... + ^ +DETAIL: Array found in unexpected place. +SELECT '[{"attributes" : 1, "dependency" : 4, "degree": 1.000}]'::pg_depen= dencies; +ERROR: malformed pg_dependencies: "[{"attributes" : 1, "dependency" : 4, = "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : 1, "dependency" : 4, "degree": 1.00... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : "a", "dependency" : 4, "degree": 1.000}]'::pg_dep= endencies; +ERROR: malformed pg_dependencies: "[{"attributes" : "a", "dependency" : 4= , "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : "a", "dependency" : 4, "degree": 1.... + ^ +DETAIL: Unexpected scalar. +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": NaN}]'::pg_dep= endencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := 4, "degree": NaN}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": ... + ^ +DETAIL: Must be valid JSON. +-- Duplicated attributes +SELECT '[{"attributes" : [2,2], "dependency" : 4, "degree": 0.500}]'::pg_d= ependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,2], "dependency" := 4, "degree": 0.500}]" +LINE 1: SELECT '[{"attributes" : [2,2], "dependency" : 4, "degree": ... + ^ +DETAIL: Duplicated attribute number found: 2. +-- Duplicated attribute lists. +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3], "dependency" : 4, "degree": 1.000}]'::pg_d= ependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := 4, "degree": 1.000}, + {"attributes" : [2,3], "dependency" : 4, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": ... + ^ +DETAIL: Duplicate "attributes" array: [2, 3] with "dependency": 4. +-- Partially-covered attribute lists. +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 1.000}, + {"attributes" : [1,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1,-2], "dependency" : 4, "degree": 1.000}]'= ::pg_dependencies; +ERROR: malformed pg_dependencies: "[{"attributes" : [2,3], "dependency" := 4, "degree": 1.000}, + {"attributes" : [1,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1,-2], "dependency" : 4, "degree": 1.000}]" +LINE 1: SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": ... + ^ +DETAIL: "attributes" array: [1, -1] with dependency 4 must be a subset of= array: [2, 3, -1, -2] with dependency 4. +-- Valid inputs +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 0.250}, + {"attributes" : [2,-1], "dependency" : 4, "degree": 0.500}, + {"attributes" : [2,3,-1], "dependency" : 4, "degree": 0.750}, + {"attributes" : [2,3,-1,-2], "dependency" : 4, "degree": 1.000}]'= ::pg_dependencies; + = pg_dependencies = = =20 +--------------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------------------------------------------------------------= ----------------------------------- + [{"attributes": [2, 3], "dependency": 4, "degree": 0.250000}, {"attribute= s": [2, -1], "dependency": 4, "degree": 0.500000}, {"attributes": [2, 3, -1= ], "dependency": 4, "degree": 0.750000}, {"attributes": [2, 3, -1, -2], "de= pendency": 4, "degree": 1.000000}] +(1 row) + diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel= _schedule index f3f0b5f2f317..cc6d799bceaf 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -28,7 +28,7 @@ test: strings md5 numerology point lseg line box path pol= ygon circle date time t # geometry depends on point, lseg, line, box, path, polygon, circle # horology depends on date, time, timetz, timestamp, timestamptz, interval # ---------- -test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity c= omments expressions unicode xid mvcc database stats_import pg_ndistinct +test: geometry horology tstypes regex type_sanity opr_sanity misc_sanity c= omments expressions unicode xid mvcc database stats_import pg_ndistinct pg_= dependencies =20 # ---------- # Load huge amounts of data diff --git a/src/test/regress/sql/pg_dependencies.sql b/src/test/regress/sq= l/pg_dependencies.sql new file mode 100644 index 000000000000..e94927f5d09d --- /dev/null +++ b/src/test/regress/sql/pg_dependencies.sql @@ -0,0 +1,47 @@ +-- Tests for type pg_distinct + +-- Invalid inputs +SELECT 'null'::pg_dependencies; +SELECT '{"a": 1}'::pg_dependencies; +SELECT '[]'::pg_dependencies; +SELECT '{}'::pg_dependencies; +SELECT '[null]'::pg_dependencies; +SELECT * FROM pg_input_error_info('null', 'pg_dependencies'); +SELECT * FROM pg_input_error_info('{"a": 1}', 'pg_dependencies'); +SELECT * FROM pg_input_error_info('[]', 'pg_dependencies'); +SELECT * FROM pg_input_error_info('{}', 'pg_dependencies'); +SELECT * FROM pg_input_error_info('[null]', 'pg_dependencies'); +-- Invalid keys +SELECT '[{"attributes_invalid" : [2,3], "dependency" : 4}]'::pg_dependenci= es; +SELECT '[{"attributes" : [2,3], "invalid" : 3, "dependency" : 4}]'::pg_dep= endencies; +-- Missing keys +SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_dependencies; +SELECT '[{"attributes" : [2,3], "degree" : 1.000}]'::pg_dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : 4}]'::pg_dependencies; +-- Valid keys, invalid values +SELECT '[{"attributes" : null, "dependency" : 4, "degree": 1.000}]'::pg_de= pendencies; +SELECT '[{"attributes" : [2,null], "dependency" : 4, "degree": 1.000}]'::p= g_dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : null, "degree": 1.000}]'::p= g_dependencies; +SELECT '[{"attributes" : [2,"a"], "dependency" : 4, "degree": 1.000}]'::pg= _dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : "a", "degree": 1.000}]'::pg= _dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : [], "degree": 1.000}]'::pg_= dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : [null], "degree": 1.000}]':= :pg_dependencies; +SELECT '[{"attributes" : [2,3], "dependency" : [1,null], "degree": 1.000}]= '::pg_dependencies; +SELECT '[{"attributes" : 1, "dependency" : 4, "degree": 1.000}]'::pg_depen= dencies; +SELECT '[{"attributes" : "a", "dependency" : 4, "degree": 1.000}]'::pg_dep= endencies; +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": NaN}]'::pg_dep= endencies; +-- Duplicated attributes +SELECT '[{"attributes" : [2,2], "dependency" : 4, "degree": 0.500}]'::pg_d= ependencies; +-- Duplicated attribute lists. +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3], "dependency" : 4, "degree": 1.000}]'::pg_d= ependencies; +-- Partially-covered attribute lists. +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 1.000}, + {"attributes" : [1,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1], "dependency" : 4, "degree": 1.000}, + {"attributes" : [2,3,-1,-2], "dependency" : 4, "degree": 1.000}]'= ::pg_dependencies; +-- Valid inputs +SELECT '[{"attributes" : [2,3], "dependency" : 4, "degree": 0.250}, + {"attributes" : [2,-1], "dependency" : 4, "degree": 0.500}, + {"attributes" : [2,3,-1], "dependency" : 4, "degree": 0.750}, + {"attributes" : [2,3,-1,-2], "dependency" : 4, "degree": 1.000}]'= ::pg_dependencies; --=20 2.51.0 --AorqrXjCfzfo1Gz1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v14-0003-Expose-attribute-statistics-functions-for-use-in.patch" Content-Transfer-Encoding: quoted-printable =46rom 34c8c39116e8ef854832419b2b287cc782f79421 Mon Sep 17 00:00:00 2001 =46rom: Corey Huinker Date: Tue, 4 Nov 2025 23:50:01 -0500 Subject: [PATCH v14 3/5] Expose attribute statistics functions for use in extended_stats. Many of the operations of attribute stats have analogous operations in extended stats. * get_attr_stat_type() renamed to statatt_get_type() * init_empty_stats_tuple() renamed to statatt_init_empty_tuple() * text_to_stavalues() * get_elem_stat_type() renamed to statatt_get_elem_type() Also, add comments explaining the function argument index enums, and the arrays that are indexed by those enums. --- src/include/statistics/statistics.h | 17 +++ src/backend/statistics/attribute_stats.c | 126 +++++++++++------------ 2 files changed, 77 insertions(+), 66 deletions(-) diff --git a/src/include/statistics/statistics.h b/src/include/statistics/s= tatistics.h index 7dd0f9755454..0df66b352a10 100644 --- a/src/include/statistics/statistics.h +++ b/src/include/statistics/statistics.h @@ -127,4 +127,21 @@ extern StatisticExtInfo *choose_best_statistics(List *= stats, char requiredkind, int nclauses); extern HeapTuple statext_expressions_load(Oid stxoid, bool inh, int idx); =20 +extern void statatt_get_type(Oid reloid, AttrNumber attnum, + Oid *atttypid, int32 *atttypmod, + char *atttyptype, Oid *atttypcoll, + Oid *eq_opr, Oid *lt_opr); +extern void statatt_init_empty_tuple(Oid reloid, int16 attnum, bool inheri= ted, + Datum *values, bool *nulls, bool *replaces); + +extern void statatt_set_slot(Datum *values, bool *nulls, bool *replaces, + int16 stakind, Oid staop, Oid stacoll, + Datum stanumbers, bool stanumbers_isnull, + Datum stavalues, bool stavalues_isnull); + +extern Datum text_to_stavalues(const char *staname, FmgrInfo *array_in, Da= tum d, + Oid typid, int32 typmod, bool *ok); +extern bool statatt_get_elem_type(Oid atttypid, char atttyptype, + Oid *elemtypid, Oid *elem_eq_opr); + #endif /* STATISTICS_H */ diff --git a/src/backend/statistics/attribute_stats.c b/src/backend/statist= ics/attribute_stats.c index ef4d768feab7..d0c67a4128e0 100644 --- a/src/backend/statistics/attribute_stats.c +++ b/src/backend/statistics/attribute_stats.c @@ -64,6 +64,10 @@ enum attribute_stats_argnum NUM_ATTRIBUTE_STATS_ARGS }; =20 +/* + * The argument names and typoids of the arguments for + * attribute_statistics_update. + */ static struct StatsArgInfo attarginfo[] =3D { [ATTRELSCHEMA_ARG] =3D {"schemaname", TEXTOID}, @@ -101,6 +105,10 @@ enum clear_attribute_stats_argnum C_NUM_ATTRIBUTE_STATS_ARGS }; =20 +/* + * The argument names and typoids of the arguments for + * pg_clear_attribute_stats. + */ static struct StatsArgInfo cleararginfo[] =3D { [C_ATTRELSCHEMA_ARG] =3D {"relation", TEXTOID}, @@ -112,23 +120,9 @@ static struct StatsArgInfo cleararginfo[] =3D =20 static bool attribute_statistics_update(FunctionCallInfo fcinfo); static Node *get_attr_expr(Relation rel, int attnum); -static void get_attr_stat_type(Oid reloid, AttrNumber attnum, - Oid *atttypid, int32 *atttypmod, - char *atttyptype, Oid *atttypcoll, - Oid *eq_opr, Oid *lt_opr); -static bool get_elem_stat_type(Oid atttypid, char atttyptype, - Oid *elemtypid, Oid *elem_eq_opr); -static Datum text_to_stavalues(const char *staname, FmgrInfo *array_in, Da= tum d, - Oid typid, int32 typmod, bool *ok); -static void set_stats_slot(Datum *values, bool *nulls, bool *replaces, - int16 stakind, Oid staop, Oid stacoll, - Datum stanumbers, bool stanumbers_isnull, - Datum stavalues, bool stavalues_isnull); static void upsert_pg_statistic(Relation starel, HeapTuple oldtup, const Datum *values, const bool *nulls, const bool *replaces); static bool delete_pg_statistic(Oid reloid, AttrNumber attnum, bool stainh= erit); -static void init_empty_stats_tuple(Oid reloid, int16 attnum, bool inherite= d, - Datum *values, bool *nulls, bool *replaces); =20 /* * Insert or Update Attribute Statistics @@ -298,16 +292,16 @@ attribute_statistics_update(FunctionCallInfo fcinfo) } =20 /* derive information from attribute */ - get_attr_stat_type(reloid, attnum, - &atttypid, &atttypmod, - &atttyptype, &atttypcoll, - &eq_opr, <_opr); + statatt_get_type(reloid, attnum, + &atttypid, &atttypmod, + &atttyptype, &atttypcoll, + &eq_opr, <_opr); =20 /* if needed, derive element type */ if (do_mcelem || do_dechist) { - if (!get_elem_stat_type(atttypid, atttyptype, - &elemtypid, &elem_eq_opr)) + if (!statatt_get_elem_type(atttypid, atttyptype, + &elemtypid, &elem_eq_opr)) { ereport(WARNING, (errmsg("could not determine element type of column \"%s\"", attname), @@ -361,7 +355,7 @@ attribute_statistics_update(FunctionCallInfo fcinfo) if (HeapTupleIsValid(statup)) heap_deform_tuple(statup, RelationGetDescr(starel), values, nulls); else - init_empty_stats_tuple(reloid, attnum, inherited, values, nulls, + statatt_init_empty_tuple(reloid, attnum, inherited, values, nulls, replaces); =20 /* if specified, set to argument values */ @@ -394,10 +388,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) =20 if (converted) { - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_MCV, - eq_opr, atttypcoll, - stanumbers, false, stavalues, false); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCV, + eq_opr, atttypcoll, + stanumbers, false, stavalues, false); } else result =3D false; @@ -417,10 +411,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) =20 if (converted) { - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_HISTOGRAM, - lt_opr, atttypcoll, - 0, true, stavalues, false); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_HISTOGRAM, + lt_opr, atttypcoll, + 0, true, stavalues, false); } else result =3D false; @@ -433,10 +427,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) ArrayType *arry =3D construct_array_builtin(elems, 1, FLOAT4OID); Datum stanumbers =3D PointerGetDatum(arry); =20 - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_CORRELATION, - lt_opr, atttypcoll, - stanumbers, false, 0, true); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_CORRELATION, + lt_opr, atttypcoll, + stanumbers, false, 0, true); } =20 /* STATISTIC_KIND_MCELEM */ @@ -454,10 +448,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) =20 if (converted) { - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_MCELEM, - elem_eq_opr, atttypcoll, - stanumbers, false, stavalues, false); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCELEM, + elem_eq_opr, atttypcoll, + stanumbers, false, stavalues, false); } else result =3D false; @@ -468,10 +462,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) { Datum stanumbers =3D PG_GETARG_DATUM(ELEM_COUNT_HISTOGRAM_ARG); =20 - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_DECHIST, - elem_eq_opr, atttypcoll, - stanumbers, false, 0, true); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_DECHIST, + elem_eq_opr, atttypcoll, + stanumbers, false, 0, true); } =20 /* @@ -494,10 +488,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) =20 if (converted) { - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_BOUNDS_HISTOGRAM, - InvalidOid, InvalidOid, - 0, true, stavalues, false); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_BOUNDS_HISTOGRAM, + InvalidOid, InvalidOid, + 0, true, stavalues, false); } else result =3D false; @@ -521,10 +515,10 @@ attribute_statistics_update(FunctionCallInfo fcinfo) =20 if (converted) { - set_stats_slot(values, nulls, replaces, - STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM, - Float8LessOperator, InvalidOid, - stanumbers, false, stavalues, false); + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM, + Float8LessOperator, InvalidOid, + stanumbers, false, stavalues, false); } else result =3D false; @@ -584,11 +578,11 @@ get_attr_expr(Relation rel, int attnum) /* * Derive type information from the attribute. */ -static void -get_attr_stat_type(Oid reloid, AttrNumber attnum, - Oid *atttypid, int32 *atttypmod, - char *atttyptype, Oid *atttypcoll, - Oid *eq_opr, Oid *lt_opr) +void +statatt_get_type(Oid reloid, AttrNumber attnum, + Oid *atttypid, int32 *atttypmod, + char *atttyptype, Oid *atttypcoll, + Oid *eq_opr, Oid *lt_opr) { Relation rel =3D relation_open(reloid, AccessShareLock); Form_pg_attribute attr; @@ -666,9 +660,9 @@ get_attr_stat_type(Oid reloid, AttrNumber attnum, /* * Derive element type information from the attribute type. */ -static bool -get_elem_stat_type(Oid atttypid, char atttyptype, - Oid *elemtypid, Oid *elem_eq_opr) +bool +statatt_get_elem_type(Oid atttypid, char atttyptype, + Oid *elemtypid, Oid *elem_eq_opr) { TypeCacheEntry *elemtypcache; =20 @@ -706,7 +700,7 @@ get_elem_stat_type(Oid atttypid, char atttyptype, * to false. If the resulting array contains NULLs, raise a WARNING and se= t ok * to false. Otherwise, set ok to true. */ -static Datum +Datum text_to_stavalues(const char *staname, FmgrInfo *array_in, Datum d, Oid ty= pid, int32 typmod, bool *ok) { @@ -759,11 +753,11 @@ text_to_stavalues(const char *staname, FmgrInfo *arra= y_in, Datum d, Oid typid, * Find and update the slot with the given stakind, or use the first empty * slot. */ -static void -set_stats_slot(Datum *values, bool *nulls, bool *replaces, - int16 stakind, Oid staop, Oid stacoll, - Datum stanumbers, bool stanumbers_isnull, - Datum stavalues, bool stavalues_isnull) +void +statatt_set_slot(Datum *values, bool *nulls, bool *replaces, + int16 stakind, Oid staop, Oid stacoll, + Datum stanumbers, bool stanumbers_isnull, + Datum stavalues, bool stavalues_isnull) { int slotidx; int first_empty =3D -1; @@ -883,9 +877,9 @@ delete_pg_statistic(Oid reloid, AttrNumber attnum, bool= stainherit) /* * Initialize values and nulls for a new stats tuple. */ -static void -init_empty_stats_tuple(Oid reloid, int16 attnum, bool inherited, - Datum *values, bool *nulls, bool *replaces) +void +statatt_init_empty_tuple(Oid reloid, int16 attnum, bool inherited, + Datum *values, bool *nulls, bool *replaces) { memset(nulls, true, sizeof(bool) * Natts_pg_statistic); memset(replaces, true, sizeof(bool) * Natts_pg_statistic); --=20 2.51.0 --AorqrXjCfzfo1Gz1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v14-0004-Add-extended-statistics-support-functions.patch" Content-Transfer-Encoding: quoted-printable =46rom 4f0a72a0a052dd337315db63839acd25544fbbf4 Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Tue, 11 Nov 2025 16:58:26 +0900 Subject: [PATCH v14 4/5] Add extended statistics support functions. Add pg_restore_extended_stats() and pg_clear_extended_stats(). These functions closely mirror their relation and attribute counterparts, but for extended statistics (i.e. CREATE STATISTICS) objects. --- src/include/catalog/pg_proc.dat | 18 + .../statistics/extended_stats_internal.h | 17 + src/backend/statistics/dependencies.c | 61 + src/backend/statistics/extended_stats.c | 1141 ++++++++++++++++- src/backend/statistics/mcv.c | 144 +++ src/backend/statistics/mvdistinct.c | 62 + src/test/regress/expected/stats_import.out | 1123 ++++++++++++++++ src/test/regress/sql/stats_import.sql | 364 ++++++ doc/src/sgml/func/func-admin.sgml | 98 ++ 9 files changed, 3027 insertions(+), 1 deletion(-) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.= dat index 5cf9e12fcb9a..84e9f176d192 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12594,6 +12594,24 @@ proname =3D> 'gist_translate_cmptype_common', prorettype =3D> 'int2', proargtypes =3D> 'int4', prosrc =3D> 'gist_translate_cmptype_common' }, =20 +# Extended Statistics Import +{ oid =3D> '9947', + descr =3D> 'restore statistics on extended statistics object', + proname =3D> 'pg_restore_extended_stats', provolatile =3D> 'v', proisstr= ict =3D> 'f', + provariadic =3D> 'any', + proparallel =3D> 'u', prorettype =3D> 'bool', + proargtypes =3D> 'any', + proargnames =3D> '{kwargs}', + proargmodes =3D> '{v}', + prosrc =3D> 'pg_restore_extended_stats' }, +{ oid =3D> '9948', + descr =3D> 'clear statistics on extended statistics object', + proname =3D> 'pg_clear_extended_stats', provolatile =3D> 'v', proisstric= t =3D> 'f', + proparallel =3D> 'u', prorettype =3D> 'void', + proargtypes =3D> 'text text bool', + proargnames =3D> '{statistics_schemaname,statistics_name,inherited}', + prosrc =3D> 'pg_clear_extended_stats' }, + # AIO related functions { oid =3D> '6399', descr =3D> 'information about in-progress asynchronous = IOs', proname =3D> 'pg_get_aios', prorows =3D> '100', proretset =3D> 't', diff --git a/src/include/statistics/extended_stats_internal.h b/src/include= /statistics/extended_stats_internal.h index efcb7dc35461..ba7f5dcad829 100644 --- a/src/include/statistics/extended_stats_internal.h +++ b/src/include/statistics/extended_stats_internal.h @@ -127,4 +127,21 @@ extern Selectivity mcv_clause_selectivity_or(PlannerIn= fo *root, Selectivity *overlap_basesel, Selectivity *totalsel); =20 +extern Datum import_mcvlist(HeapTuple tup, int elevel, int numattrs, + Oid *atttypids, int32 *atttypmods, Oid *atttypcolls, + int nitems, Datum *mcv_elems, bool *mcv_nulls, + bool *mcv_elem_nulls, float8 *freqs, float8 *base_freqs); + +extern Datum import_mcvlist(HeapTuple tup, int elevel, int numattrs, + Oid *atttypids, int32 *atttypmods, Oid *atttypcolls, + int nitems, Datum *mcv_elems, bool *mcv_nulls, + bool *mcv_elem_nulls, float8 *freqs, float8 *base_freqs); +extern bool pg_ndistinct_validate_items(MVNDistinct *ndistinct, int2vector= *stxkeys, + int numexprs, int elevel); +extern void free_pg_ndistinct(MVNDistinct *ndistinct); +extern bool pg_dependencies_validate_deps(MVDependencies *dependencies, + int2vector *stxkeys, int numexprs, + int elevel); +extern void free_pg_dependencies(MVDependencies *dependencies); + #endif /* EXTENDED_STATS_INTERNAL_H */ diff --git a/src/backend/statistics/dependencies.c b/src/backend/statistics= /dependencies.c index 6f63b4f3ffbf..31a9f1cfc7c9 100644 --- a/src/backend/statistics/dependencies.c +++ b/src/backend/statistics/dependencies.c @@ -1065,6 +1065,55 @@ clauselist_apply_dependencies(PlannerInfo *root, Lis= t *clauses, return s1; } =20 +/* + * Validate an MVDependencies against the extended statistics object defin= ition. + * + * Every MVDependencies must be checked to ensure that the attnums in the + * attributes list correspond to attnums/expressions defined by the + * extended statistics object. + * + * Positive attnums are attributes which must be found in the stxkeys, + * while negative attnums correspond to an expr number, so the attnum + * can't be below (0 - numexprs). + */ +bool +pg_dependencies_validate_deps(MVDependencies *dependencies, int2vector *st= xkeys, int numexprs, int elevel) +{ + int attnum_expr_lowbound =3D 0 - numexprs; + + for (int i =3D 0; i < dependencies->ndeps; i++) + { + MVDependency *dep =3D dependencies->deps[i]; + + for (int j =3D 0; j < dep->nattributes; j++) + { + AttrNumber attnum =3D dep->attributes[j]; + bool ok =3D false; + + if (attnum > 0) + { + for (int k =3D 0; k < stxkeys->dim1; k++) + if (attnum =3D=3D stxkeys->values[k]) + { + ok =3D true; + break; + } + } + else if ((attnum < 0) && (attnum >=3D attnum_expr_lowbound)) + ok =3D true; + + if (!ok) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("pg_dependencies: invalid attnum for this statistics object: = %d", attnum))); + return false; + } + } + } + return true; +} + /* * dependency_is_compatible_expression * Determines if the expression is compatible with functional dependencies @@ -1248,6 +1297,18 @@ dependency_is_compatible_expression(Node *clause, In= dex relid, List *statlist, N return false; } =20 +/* + * Free allocations of an MVNDistinct + */ +void +free_pg_dependencies(MVDependencies *dependencies) +{ + for (int i =3D 0; i < dependencies->ndeps; i++) + pfree(dependencies->deps[i]); + + pfree(dependencies); +} + /* * dependencies_clauselist_selectivity * Return the estimated selectivity of (a subset of) the given clauses diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statisti= cs/extended_stats.c index 3c3d2d315c6f..23ab3cf87e18 100644 --- a/src/backend/statistics/extended_stats.c +++ b/src/backend/statistics/extended_stats.c @@ -18,21 +18,28 @@ =20 #include "access/detoast.h" #include "access/genam.h" +#include "access/heapam.h" +#include "access/htup.h" #include "access/htup_details.h" #include "access/table.h" #include "catalog/indexing.h" +#include "catalog/pg_collation.h" #include "catalog/pg_statistic_ext.h" #include "catalog/pg_statistic_ext_data.h" +#include "catalog/pg_type_d.h" +#include "catalog/namespace.h" #include "commands/defrem.h" #include "commands/progress.h" #include "executor/executor.h" #include "miscadmin.h" +#include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" #include "optimizer/optimizer.h" #include "parser/parsetree.h" #include "pgstat.h" #include "postmaster/autovacuum.h" #include "statistics/extended_stats_internal.h" +#include "statistics/stat_utils.h" #include "statistics/statistics.h" #include "utils/acl.h" #include "utils/array.h" @@ -72,6 +79,84 @@ typedef struct StatExtEntry List *exprs; /* expressions */ } StatExtEntry; =20 +/* + * An index of the args for extended_statistics_update(). + */ +enum extended_stats_argnum +{ + STATSCHEMA_ARG =3D 0, + STATNAME_ARG, + INHERITED_ARG, + NDISTINCT_ARG, + DEPENDENCIES_ARG, + MOST_COMMON_VALS_ARG, + MOST_COMMON_VAL_NULLS_ARG, + MOST_COMMON_FREQS_ARG, + MOST_COMMON_BASE_FREQS_ARG, + EXPRESSIONS_ARG, + NUM_EXTENDED_STATS_ARGS +}; + +/* + * The argument names and typoids of the arguments for + * extended_statistics_update(). + */ +static struct StatsArgInfo extarginfo[] =3D +{ + [STATSCHEMA_ARG] =3D {"statistics_schemaname", TEXTOID}, + [STATNAME_ARG] =3D {"statistics_name", TEXTOID}, + [INHERITED_ARG] =3D {"inherited", BOOLOID}, + [NDISTINCT_ARG] =3D {"n_distinct", PG_NDISTINCTOID}, + [DEPENDENCIES_ARG] =3D {"dependencies", PG_DEPENDENCIESOID}, + [MOST_COMMON_VALS_ARG] =3D {"most_common_vals", TEXTARRAYOID}, + [MOST_COMMON_VAL_NULLS_ARG] =3D {"most_common_val_nulls", BOOLARRAYOID}, + [MOST_COMMON_FREQS_ARG] =3D {"most_common_freqs", FLOAT8ARRAYOID}, + [MOST_COMMON_BASE_FREQS_ARG] =3D {"most_common_base_freqs", FLOAT8ARRAYOI= D}, + [EXPRESSIONS_ARG] =3D {"exprs", TEXTARRAYOID}, + [NUM_EXTENDED_STATS_ARGS] =3D {0} +}; + +/* + * An index of the elements of the stxdexprs datum, which repeat for each + * expression in the extended statistics object. + * + * NOTE: the RANGE_LENGTH & RANGE_BOUNDS stats are not yet reflected in any + * version of pg_stat_ext_exprs. + */ +enum extended_stats_exprs_element +{ + NULL_FRAC_ELEM =3D 0, + AVG_WIDTH_ELEM, + N_DISTINCT_ELEM, + MOST_COMMON_VALS_ELEM, + MOST_COMMON_FREQS_ELEM, + HISTOGRAM_BOUNDS_ELEM, + CORRELATION_ELEM, + MOST_COMMON_ELEMS_ELEM, + MOST_COMMON_ELEM_FREQS_ELEM, + ELEM_COUNT_HISTOGRAM_ELEM, + NUM_ATTRIBUTE_STATS_ELEMS +}; + +/* + * The argument names and typoids of the repeating arguments for stxdexprs. + */ +static struct StatsArgInfo extexprarginfo[] =3D +{ + [NULL_FRAC_ELEM] =3D {"null_frac", FLOAT4OID}, + [AVG_WIDTH_ELEM] =3D {"avg_width", INT4OID}, + [N_DISTINCT_ELEM] =3D {"n_distinct", FLOAT4OID}, + [MOST_COMMON_VALS_ELEM] =3D {"most_common_vals", TEXTOID}, + [MOST_COMMON_FREQS_ELEM] =3D {"most_common_freqs", FLOAT4ARRAYOID}, + [HISTOGRAM_BOUNDS_ELEM] =3D {"histogram_bounds", TEXTOID}, + [CORRELATION_ELEM] =3D {"correlation", FLOAT4OID}, + [MOST_COMMON_ELEMS_ELEM] =3D {"most_common_elems", TEXTOID}, + [MOST_COMMON_ELEM_FREQS_ELEM] =3D {"most_common_elem_freqs", FLOAT4ARRAYO= ID}, + [ELEM_COUNT_HISTOGRAM_ELEM] =3D {"elem_count_histogram", FLOAT4ARRAYOID}, + [NUM_ATTRIBUTE_STATS_ELEMS] =3D {0} +}; + +static bool extended_statistics_update(FunctionCallInfo fcinfo); =20 static List *fetch_statentries_for_relation(Relation pg_statext, Oid relid= ); static VacAttrStats **lookup_var_attr_stats(Bitmapset *attrs, List *exprs, @@ -99,6 +184,28 @@ static StatsBuildData *make_build_data(Relation rel, St= atExtEntry *stat, int numrows, HeapTuple *rows, VacAttrStats **stats, int stattarget); =20 +static HeapTuple get_pg_statistic_ext(Relation pg_stext, Oid nspoid, + const char *stxname); +static bool delete_pg_statistic_ext_data(Oid stxoid, bool inherited); + +typedef struct +{ + bool ndistinct; + bool dependencies; + bool mcv; + bool expressions; +} stakindFlags; + +static void expand_stxkind(HeapTuple tup, stakindFlags * enabled); +static void upsert_pg_statistic_ext_data(Datum *values, bool *nulls, bool = *replaces); +static bool check_mcvlist_array(ArrayType *arr, int argindex, + int required_ndims, int mcv_length); +static Datum import_expressions(Relation pgsd, int numexprs, + Oid *atttypids, int32 *atttypmods, + Oid *atttypcolls, ArrayType *exprs_arr); +static bool text_to_float4(Datum input, Datum *output); +static bool text_to_int4(Datum input, Datum *output); + =20 /* * Compute requested extended stats, using the rows sampled for the plain @@ -121,7 +228,7 @@ BuildRelationExtStatistics(Relation onerel, bool inh, d= ouble totalrows, =20 /* Do nothing if there are no columns to analyze. */ if (!natts) - return; + return; =20 /* the list of stats has to be allocated outside the memory context */ pg_stext =3D table_open(StatisticExtRelationId, RowExclusiveLock); @@ -2612,3 +2719,1035 @@ make_build_data(Relation rel, StatExtEntry *stat, = int numrows, HeapTuple *rows, =20 return result; } + +static HeapTuple +get_pg_statistic_ext(Relation pg_stext, Oid nspoid, const char *stxname) +{ + ScanKeyData key[2]; + SysScanDesc scan; + HeapTuple tup; + Oid stxoid =3D InvalidOid; + + ScanKeyInit(&key[0], + Anum_pg_statistic_ext_stxname, + BTEqualStrategyNumber, + F_NAMEEQ, + CStringGetDatum(stxname)); + ScanKeyInit(&key[1], + Anum_pg_statistic_ext_stxnamespace, + BTEqualStrategyNumber, + F_OIDEQ, + ObjectIdGetDatum(nspoid)); + + /* + * Try to find matching pg_statistic_ext row. + */ + scan =3D systable_beginscan(pg_stext, + StatisticExtNameIndexId, + true, + NULL, + 2, + key); + + /* Unique index, either we get a tuple or we don't. */ + tup =3D systable_getnext(scan); + + if (HeapTupleIsValid(tup)) + stxoid =3D ((Form_pg_statistic_ext) GETSTRUCT(tup))->oid; + + systable_endscan(scan); + + if (!OidIsValid(stxoid)) + return NULL; + + return SearchSysCacheCopy1(STATEXTOID, ObjectIdGetDatum(stxoid)); +} + +/* + * Decode the stxkind column so that we know which stats types to expect. + */ +static void +expand_stxkind(HeapTuple tup, stakindFlags * enabled) +{ + Datum datum; + ArrayType *arr; + char *kinds; + + datum =3D SysCacheGetAttrNotNull(STATEXTOID, + tup, + Anum_pg_statistic_ext_stxkind); + arr =3D DatumGetArrayTypeP(datum); + if (ARR_NDIM(arr) !=3D 1 || ARR_HASNULL(arr) || ARR_ELEMTYPE(arr) !=3D CH= AROID) + elog(ERROR, "stxkind is not a 1-D char array"); + + kinds =3D (char *) ARR_DATA_PTR(arr); + + for (int i =3D 0; i < ARR_DIMS(arr)[0]; i++) + if (kinds[i] =3D=3D STATS_EXT_NDISTINCT) + enabled->ndistinct =3D true; + else if (kinds[i] =3D=3D STATS_EXT_DEPENDENCIES) + enabled->dependencies =3D true; + else if (kinds[i] =3D=3D STATS_EXT_MCV) + enabled->mcv =3D true; + else if (kinds[i] =3D=3D STATS_EXT_EXPRESSIONS) + enabled->expressions =3D true; +} + +static void +upsert_pg_statistic_ext_data(Datum *values, bool *nulls, bool *replaces) +{ + Relation pg_stextdata; + HeapTuple stxdtup; + HeapTuple newtup; + + pg_stextdata =3D table_open(StatisticExtDataRelationId, RowExclusiveLock); + + stxdtup =3D SearchSysCache2(STATEXTDATASTXOID, + values[Anum_pg_statistic_ext_data_stxoid - 1], + values[Anum_pg_statistic_ext_data_stxdinherit - 1]); + + if (HeapTupleIsValid(stxdtup)) + { + newtup =3D heap_modify_tuple(stxdtup, + RelationGetDescr(pg_stextdata), + values, + nulls, + replaces); + CatalogTupleUpdate(pg_stextdata, &newtup->t_self, newtup); + ReleaseSysCache(stxdtup); + } + else + { + newtup =3D heap_form_tuple(RelationGetDescr(pg_stextdata), values, nulls= ); + CatalogTupleInsert(pg_stextdata, newtup); + } + + heap_freetuple(newtup); + + CommandCounterIncrement(); + + table_close(pg_stextdata, RowExclusiveLock); +} + +/* + * Insert or Update Extended Statistics + * + * Major errors, such as the table not existing, the statistics object not + * existing, or a permissions failure are always reported at ERROR. Other + * errors, such as a conversion failure on one statistic kind, are reported + * as WARNINGs, and other statistic kinds may still be updated. + */ +static bool +extended_statistics_update(FunctionCallInfo fcinfo) +{ + Oid nspoid; + char *nspname; + char *stxname; + bool inherited; + Relation pg_stext; + HeapTuple tup =3D NULL; + + stakindFlags enabled; + stakindFlags has; + + Form_pg_statistic_ext stxform; + + Datum values[Natts_pg_statistic_ext_data]; + bool nulls[Natts_pg_statistic_ext_data]; + bool replaces[Natts_pg_statistic_ext_data]; + + bool success =3D true; + + Datum exprdatum; + bool isnull; + List *exprs =3D NIL; + int numattnums =3D 0; + int numexprs =3D 0; + int numattrs =3D 0; + + /* arrays of type info, if we need them */ + Oid *atttypids =3D NULL; + int32 *atttypmods =3D NULL; + Oid *atttypcolls =3D NULL; + Relation rel; + Oid locked_table =3D InvalidOid; + + memset(nulls, false, sizeof(nulls)); + memset(values, 0, sizeof(values)); + memset(replaces, 0, sizeof(replaces)); + memset(&enabled, 0, sizeof(enabled)); + + has.mcv =3D (!PG_ARGISNULL(MOST_COMMON_VALS_ARG) && + !PG_ARGISNULL(MOST_COMMON_VAL_NULLS_ARG) && + !PG_ARGISNULL(MOST_COMMON_FREQS_ARG) && + !PG_ARGISNULL(MOST_COMMON_BASE_FREQS_ARG)); + has.ndistinct =3D !PG_ARGISNULL(NDISTINCT_ARG); + has.dependencies =3D !PG_ARGISNULL(DEPENDENCIES_ARG); + has.expressions =3D !PG_ARGISNULL(EXPRESSIONS_ARG); + + if (RecoveryInProgress()) + { + ereport(WARNING, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("Statistics cannot be modified during recovery."))); + PG_RETURN_BOOL(false); + } + + stats_check_required_arg(fcinfo, extarginfo, STATSCHEMA_ARG); + nspname =3D TextDatumGetCString(PG_GETARG_DATUM(STATSCHEMA_ARG)); + stats_check_required_arg(fcinfo, extarginfo, STATNAME_ARG); + stxname =3D TextDatumGetCString(PG_GETARG_DATUM(STATNAME_ARG)); + stats_check_required_arg(fcinfo, extarginfo, INHERITED_ARG); + inherited =3D PG_GETARG_NAME(INHERITED_ARG); + + nspoid =3D get_namespace_oid(nspname, true); + if (nspoid =3D=3D InvalidOid) + { + ereport(WARNING, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Namespace \"%s\" not found.", stxname))); + PG_RETURN_BOOL(false); + } + + pg_stext =3D table_open(StatisticExtRelationId, RowExclusiveLock); + tup =3D get_pg_statistic_ext(pg_stext, nspoid, stxname); + + if (!HeapTupleIsValid(tup)) + { + table_close(pg_stext, RowExclusiveLock); + ereport(WARNING, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Extended Statistics Object \"%s\".\"%s\" not found.", + get_namespace_name(nspoid), stxname))); + PG_RETURN_BOOL(false); + } + + stxform =3D (Form_pg_statistic_ext) GETSTRUCT(tup); + expand_stxkind(tup, &enabled); + numattnums =3D stxform->stxkeys.dim1; + + /* decode expression (if any) */ + exprdatum =3D SysCacheGetAttr(STATEXTOID, + tup, + Anum_pg_statistic_ext_stxexprs, + &isnull); + + if (!isnull) + { + char *s; + + s =3D TextDatumGetCString(exprdatum); + exprs =3D (List *) stringToNode(s); + pfree(s); + + /* + * Run the expressions through eval_const_expressions. This is not + * just an optimization, but is necessary, because the planner + * will be comparing them to similarly-processed qual clauses, and + * may fail to detect valid matches without this. We must not use + * canonicalize_qual, however, since these aren't qual + * expressions. + */ + exprs =3D (List *) eval_const_expressions(NULL, (Node *) exprs); + + /* May as well fix opfuncids too */ + fix_opfuncids((Node *) exprs); + } + numexprs =3D list_length(exprs); + numattrs =3D numattnums + numexprs; + + /* Roundabout way of getting a RangeVar on the underlying table */ + rel =3D relation_open(stxform->stxrelid, AccessShareLock); + + /* no need to fetch reloid, we already have it */ + RangeVarGetRelidExtended(makeRangeVar(nspname, + RelationGetRelationName(rel), -1), + ShareUpdateExclusiveLock, 0, + RangeVarCallbackForStats, &locked_table); + + relation_close(rel, AccessShareLock); + + if (has.mcv) + { + if (!enabled.mcv) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV parameters \"%s\", \"%s\", \"%s\", and \"%s\" were all " + "specified for extended statistics object that does not expect MCV = ", + extarginfo[MOST_COMMON_VALS_ARG].argname, + extarginfo[MOST_COMMON_VAL_NULLS_ARG].argname, + extarginfo[MOST_COMMON_FREQS_ARG].argname, + extarginfo[MOST_COMMON_BASE_FREQS_ARG].argname))); + has.mcv =3D false; + success =3D false; + } + } + else + { + /* The MCV args must all be NULL */ + if (!PG_ARGISNULL(MOST_COMMON_VALS_ARG) || + !PG_ARGISNULL(MOST_COMMON_VAL_NULLS_ARG) || + !PG_ARGISNULL(MOST_COMMON_FREQS_ARG) || + !PG_ARGISNULL(MOST_COMMON_BASE_FREQS_ARG)) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV parameters \"%s\", \"%s\", \"%s\", and \"%s\" must be al= l specified if any are specified", + extarginfo[MOST_COMMON_VALS_ARG].argname, + extarginfo[MOST_COMMON_VAL_NULLS_ARG].argname, + extarginfo[MOST_COMMON_FREQS_ARG].argname, + extarginfo[MOST_COMMON_BASE_FREQS_ARG].argname))); + success =3D false; + } + } + + if (has.ndistinct && !enabled.ndistinct) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics objec= t " + "that does not expect \"%s\"", + extarginfo[NDISTINCT_ARG].argname, + extarginfo[NDISTINCT_ARG].argname))); + has.ndistinct =3D false; + success =3D false; + } + + if (has.dependencies && !enabled.dependencies) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics objec= t " + "that does not expect \"%s\"", + extarginfo[DEPENDENCIES_ARG].argname, + extarginfo[DEPENDENCIES_ARG].argname))); + has.dependencies =3D false; + success =3D false; + } + + if (has.expressions && !enabled.expressions) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" was specified for extended statistics objec= t " + "that does not expect \"%s\"", + extarginfo[DEPENDENCIES_ARG].argname, + extarginfo[DEPENDENCIES_ARG].argname))); + has.expressions =3D false; + success =3D false; + } + + /* + * Either of these statsistic types requires that we supply + * semi-filled-out VacAttrStatP array. + * + * + * It is not possible to use the existing lookup_var_attr_stats() and + * examine_attribute() because these functions will skip attributes for + * which attstattarget is 0, and we may have stats to import for those + * attributes. + */ + if (has.mcv || has.expressions) + { + atttypids =3D palloc0(numattrs * sizeof(Oid)); + atttypmods =3D palloc0(numattrs * sizeof(int32)); + atttypcolls =3D palloc0(numattrs * sizeof(Oid)); + + for (int i =3D 0; i < numattnums; i++) + { + AttrNumber attnum =3D stxform->stxkeys.values[i]; + + Oid lt_opr; + Oid eq_opr; + char typetype; + + /* + * fetch attribute entries the same as are done for attribute + * stats + */ + statatt_get_type(stxform->stxrelid, + attnum, + &atttypids[i], + &atttypmods[i], + &typetype, + &atttypcolls[i], + <_opr, + &eq_opr); + } + + for (int i =3D numattnums; i < numattrs; i++) + { + Node *expr =3D list_nth(exprs, i - numattnums); + + atttypids[i] =3D exprType(expr); + atttypmods[i] =3D exprTypmod(expr); + atttypcolls[i] =3D exprCollation(expr); + + /* + * Duplicate logic from get_attr_stat_type + */ + + /* + * If it's a multirange, step down to the range type, as is done + * by multirange_typanalyze(). + */ + if (type_is_multirange(atttypids[i])) + atttypids[i] =3D get_multirange_range(atttypids[i]); + + /* + * Special case: collation for tsvector is DEFAULT_COLLATION_OID. + * See compute_tsvector_stats(). + */ + if (atttypids[i] =3D=3D TSVECTOROID) + atttypcolls[i] =3D DEFAULT_COLLATION_OID; + + } + } + + /* Primary Key: cannot be NULL or replaced. */ + values[Anum_pg_statistic_ext_data_stxoid - 1] =3D ObjectIdGetDatum(stxfor= m->oid); + values[Anum_pg_statistic_ext_data_stxdinherit - 1] =3D BoolGetDatum(inher= ited); + + if (has.ndistinct) + { + Datum ndistinct_datum =3D PG_GETARG_DATUM(NDISTINCT_ARG); + bytea *data =3D DatumGetByteaPP(ndistinct_datum); + MVNDistinct *ndistinct =3D statext_ndistinct_deserialize(data); + + if (pg_ndistinct_validate_items(ndistinct, &stxform->stxkeys, numexprs, = WARNING)) + { + values[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D ndistinct_datu= m; + replaces[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + } + else + { + nulls[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + success =3D false; + } + + free_pg_ndistinct(ndistinct); + } + else + nulls[Anum_pg_statistic_ext_data_stxdndistinct - 1] =3D true; + + if (has.dependencies) + { + Datum dependencies_datum =3D PG_GETARG_DATUM(DEPENDENCIES_ARG); + bytea *data =3D DatumGetByteaPP(dependencies_datum); + MVDependencies *dependencies =3D statext_dependencies_deserialize(data); + + if (pg_dependencies_validate_deps(dependencies, &stxform->stxkeys, numex= prs, WARNING)) + { + values[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D dependencie= s_datum; + replaces[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + } + else + { + nulls[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + success =3D false; + } + + free_pg_dependencies(dependencies); + } + else + nulls[Anum_pg_statistic_ext_data_stxddependencies - 1] =3D true; + + if (has.mcv) + { + Datum datum; + ArrayType *mcv_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_VALS_ARG); + ArrayType *nulls_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_VAL_NULLS_AR= G); + ArrayType *freqs_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_FREQS_ARG); + ArrayType *base_freqs_arr =3D PG_GETARG_ARRAYTYPE_P(MOST_COMMON_BASE_FR= EQS_ARG); + int nitems; + Datum *mcv_elems; + bool *mcv_nulls; + int check_nummcv; + + /* + * The mcv_arr is an array of arrays of text, and we use it as the + * reference array for checking the lengths of the other 3 arrays. + */ + if (ARR_NDIM(mcv_arr) !=3D 2) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" must be a text array of 2 dimensions.", + extarginfo[MOST_COMMON_VALS_ARG].argname))); + return (Datum) 0; + } + + nitems =3D ARR_DIMS(mcv_arr)[0]; + + /* fixed length arrays that cannot contain NULLs */ + if (!check_mcvlist_array(nulls_arr, MOST_COMMON_VAL_NULLS_ARG, + 2, nitems) || + !check_mcvlist_array(freqs_arr, MOST_COMMON_FREQS_ARG, + 1, nitems) || + !check_mcvlist_array(base_freqs_arr, MOST_COMMON_BASE_FREQS_ARG, + 1, nitems)) + return (Datum) 0; + + + deconstruct_array_builtin(mcv_arr, TEXTOID, &mcv_elems, + &mcv_nulls, &check_nummcv); + + Assert(check_nummcv =3D=3D (nitems * numattrs)); + + datum =3D import_mcvlist(tup, WARNING, numattrs, + atttypids, atttypmods, atttypcolls, + nitems, mcv_elems, mcv_nulls, + (bool *) ARR_DATA_PTR(nulls_arr), + (float8 *) ARR_DATA_PTR(freqs_arr), + (float8 *) ARR_DATA_PTR(base_freqs_arr)); + + values[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D datum; + replaces[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D true; + } + else + nulls[Anum_pg_statistic_ext_data_stxdmcv - 1] =3D true; + + if (has.expressions) + { + Datum datum; + Relation pgsd; + + pgsd =3D table_open(StatisticRelationId, RowExclusiveLock); + + datum =3D import_expressions(pgsd, numexprs, + &atttypids[numattnums], &atttypmods[numattnums], + &atttypcolls[numattnums], + PG_GETARG_ARRAYTYPE_P(EXPRESSIONS_ARG)); + + table_close(pgsd, RowExclusiveLock); + + values[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D datum; + replaces[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D true; + } + else + nulls[Anum_pg_statistic_ext_data_stxdexpr - 1] =3D true; + + upsert_pg_statistic_ext_data(values, nulls, replaces); + + heap_freetuple(tup); + table_close(pg_stext, RowExclusiveLock); + + if (atttypids !=3D NULL) + pfree(atttypids); + if (atttypmods !=3D NULL) + pfree(atttypmods); + if (atttypcolls !=3D NULL) + pfree(atttypcolls); + return success; +} + +/* + * Consistency checks to ensure that other mcvlist arrays are in alignment + * with the mcv array. + */ +static bool +check_mcvlist_array(ArrayType *arr, int argindex, int required_ndims, + int mcv_length) +{ + if (ARR_NDIM(arr) !=3D required_ndims) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must be an array of %d dimensions.", + extarginfo[argindex].argname, required_ndims))); + return false; + } + + if (array_contains_nulls(arr)) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Array \"%s\" cannot contain NULLs.", + extarginfo[argindex].argname))); + return false; + } + + if (ARR_DIMS(arr)[0] !=3D mcv_length) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameters \"%s\" must have the same number of elements as \"= %s\"", + extarginfo[argindex].argname, + extarginfo[MOST_COMMON_VALS_ARG].argname))); + return false; + } + + return true; +} + +/* + * Create the stxdexprs datum using the user input in an array of array of + * text, referenced against the datatypes for the expressions. + */ +static Datum +import_expressions(Relation pgsd, int numexprs, + Oid *atttypids, int32 *atttypmods, + Oid *atttypcolls, ArrayType *exprs_arr) +{ + Datum *exprs_elems; + bool *exprs_nulls; + int check_numexprs; + int offset =3D 0; + + FmgrInfo array_in_fn; + + Oid pgstypoid =3D get_rel_type_id(StatisticRelationId); + + ArrayBuildState *astate =3D NULL; + + + if (ARR_NDIM(exprs_arr) !=3D 2) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must be a text array of 2 dimensions.", + extarginfo[EXPRESSIONS_ARG].argname))); + return (Datum) 0; + } + + if (ARR_DIMS(exprs_arr)[0] !=3D numexprs) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must have an outer dimension of %d elements.= ", + extarginfo[EXPRESSIONS_ARG].argname, numexprs))); + return (Datum) 0; + } + if (ARR_DIMS(exprs_arr)[1] !=3D NUM_ATTRIBUTE_STATS_ELEMS) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Parameter \"%s\" must have an inner dimension of %d elements.= ", + extarginfo[EXPRESSIONS_ARG].argname, + NUM_ATTRIBUTE_STATS_ELEMS))); + return (Datum) 0; + } + + fmgr_info(F_ARRAY_IN, &array_in_fn); + + deconstruct_array_builtin(exprs_arr, TEXTOID, &exprs_elems, + &exprs_nulls, &check_numexprs); + + for (int i =3D 0; i < numexprs; i++) + { + Oid typid =3D atttypids[i]; + int32 typmod =3D atttypmods[i]; + Oid stacoll =3D atttypcolls[i]; + TypeCacheEntry *typcache; + + Oid elemtypid =3D InvalidOid; + Oid elem_eq_opr =3D InvalidOid; + + bool ok; + + Datum values[Natts_pg_statistic]; + bool nulls[Natts_pg_statistic]; + bool replaces[Natts_pg_statistic]; + + HeapTuple pgstup; + Datum pgstdat; + + /* finds the right operators even if atttypid is a domain */ + typcache =3D lookup_type_cache(typid, TYPECACHE_LT_OPR | TYPECACHE_EQ_OP= R); + + statatt_init_empty_tuple(InvalidOid, InvalidAttrNumber, false, + values, nulls, replaces); + + if (!exprs_nulls[offset + NULL_FRAC_ELEM]) + { + ok =3D text_to_float4(exprs_elems[offset + NULL_FRAC_ELEM], + &values[Anum_pg_statistic_stanullfrac - 1]); + + if (!ok) + { + char *s =3D TextDatumGetCString(exprs_elems[offset + NULL_FRAC_ELEM= ]); + + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input = type.", + extexprarginfo[NULL_FRAC_ELEM].argname, s))); + pfree(s); + return (Datum) 0; + } + } + + if (!exprs_nulls[offset + AVG_WIDTH_ELEM]) + { + ok =3D text_to_int4(exprs_elems[offset + AVG_WIDTH_ELEM], + &values[Anum_pg_statistic_stawidth - 1]); + + if (!ok) + { + char *s =3D TextDatumGetCString(exprs_elems[offset + NULL_FRAC_ELEM= ]); + + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input = type.", + extexprarginfo[AVG_WIDTH_ELEM].argname, s))); + pfree(s); + return (Datum) 0; + } + } + + if (!exprs_nulls[offset + N_DISTINCT_ELEM]) + { + ok =3D text_to_float4(exprs_elems[offset + N_DISTINCT_ELEM], + &values[Anum_pg_statistic_stadistinct - 1]); + + if (!ok) + { + char *s =3D TextDatumGetCString(exprs_elems[offset + NULL_FRAC_ELEM= ]); + + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input = type.", + extexprarginfo[N_DISTINCT_ELEM].argname, s))); + pfree(s); + return (Datum) 0; + } + } + + /* + * The STAKIND statistics are the same as the ones found in attribute + * stats. However, these are all derived from text columns, whereas + * the ones derived for attribute stats are a mix of datatypes. This + * limits the opportunities for code sharing between the two. + */ + + /* STATISTIC_KIND_MCV */ + if (exprs_nulls[offset + MOST_COMMON_VALS_ELEM] !=3D + exprs_nulls[offset + MOST_COMMON_FREQS_ELEM]) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s and %s must both be NOT NULL or both NULL.", + extexprarginfo[MOST_COMMON_VALS_ELEM].argname, + extexprarginfo[MOST_COMMON_FREQS_ELEM].argname))); + return (Datum) 0; + } + + if (!exprs_nulls[offset + MOST_COMMON_VALS_ELEM]) + { + Datum stavalues; + Datum stanumbers; + + stavalues =3D text_to_stavalues(extexprarginfo[MOST_COMMON_VALS_ELEM].a= rgname, + &array_in_fn, exprs_elems[offset + MOST_COMMON_VALS_ELEM], + typid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + stanumbers =3D text_to_stavalues(extexprarginfo[MOST_COMMON_VALS_ELEM].= argname, + &array_in_fn, exprs_elems[offset + MOST_COMMON_FREQS_ELEM], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCV, + typcache->eq_opr, stacoll, + stanumbers, false, stavalues, false); + } + + /* STATISTIC_KIND_HISTOGRAM */ + if (!exprs_nulls[offset + HISTOGRAM_BOUNDS_ELEM]) + { + Datum stavalues; + + stavalues =3D text_to_stavalues(extexprarginfo[HISTOGRAM_BOUNDS_ELEM].a= rgname, + &array_in_fn, exprs_elems[offset + HISTOGRAM_BOUNDS_ELEM], + typid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_HISTOGRAM, + typcache->lt_opr, stacoll, + 0, true, stavalues, false); + } + + /* STATISTIC_KIND_CORRELATION */ + if (!exprs_nulls[offset + CORRELATION_ELEM]) + { + Datum corr[] =3D {(Datum) 0}; + ArrayType *arry; + Datum stanumbers; + + ok =3D text_to_float4(exprs_elems[offset + CORRELATION_ELEM], &corr[0]); + + if (!ok) + { + char *s =3D TextDatumGetCString(exprs_elems[offset + CORRELATION_EL= EM]); + + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s element \"%s\" does not match expected input = type.", + extexprarginfo[CORRELATION_ELEM].argname, s))); + return (Datum) 0; + } + + arry =3D construct_array_builtin(corr, 1, FLOAT4OID); + + stanumbers =3D PointerGetDatum(arry); + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_CORRELATION, + typcache->lt_opr, stacoll, + stanumbers, false, 0, true); + } + + /* STATISTIC_KIND_MCELEM */ + if (exprs_nulls[offset + MOST_COMMON_ELEMS_ELEM] !=3D + exprs_nulls[offset + MOST_COMMON_ELEM_FREQS_ELEM]) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Expression %s and %s must both be NOT NULL or both NULL.", + extexprarginfo[MOST_COMMON_ELEMS_ELEM].argname, + extexprarginfo[MOST_COMMON_ELEM_FREQS_ELEM].argname))); + return (Datum) 0; + } + + /* + * We only need to fetch element type and eq operator if we have a + * stat of type MCELEM or DECHIST. + */ + if (!exprs_nulls[offset + MOST_COMMON_ELEMS_ELEM] || + !exprs_nulls[offset + ELEM_COUNT_HISTOGRAM_ELEM]) + { + if (!statatt_get_elem_type(typid, typcache->typtype, + &elemtypid, &elem_eq_opr)) + { + ereport(WARNING, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + (errmsg("unable to determine element type of expression")))); + return (Datum) 0; + } + } + + if (!exprs_nulls[offset + MOST_COMMON_ELEMS_ELEM]) + { + Datum stavalues; + Datum stanumbers; + + stavalues =3D text_to_stavalues(extexprarginfo[MOST_COMMON_ELEMS_ELEM].= argname, + &array_in_fn, + exprs_elems[offset + MOST_COMMON_ELEMS_ELEM], + elemtypid, typmod, &ok); + + if (!ok) + return (Datum) 0; + + stanumbers =3D text_to_stavalues(extexprarginfo[MOST_COMMON_ELEM_FREQS_= ELEM].argname, + &array_in_fn, + exprs_elems[offset + MOST_COMMON_ELEM_FREQS_ELEM], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, + STATISTIC_KIND_MCELEM, + elem_eq_opr, stacoll, + stanumbers, false, stavalues, false); + } + + if (!exprs_nulls[offset + ELEM_COUNT_HISTOGRAM_ELEM]) + { + Datum stanumbers; + + stanumbers =3D text_to_stavalues(extexprarginfo[ELEM_COUNT_HISTOGRAM_EL= EM].argname, + &array_in_fn, + exprs_elems[offset + ELEM_COUNT_HISTOGRAM_ELEM], + FLOAT4OID, -1, &ok); + + if (!ok) + return (Datum) 0; + + statatt_set_slot(values, nulls, replaces, STATISTIC_KIND_DECHIST, + elem_eq_opr, stacoll, + stanumbers, false, 0, true); + } + + /* + * Currently there are no extended stats exports of the statistic + * kinds STATISTIC_KIND_BOUNDS_HISTOGRAM or + * STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM so these cannot be imported. + * These may be added in the future. + */ + + pgstup =3D heap_form_tuple(RelationGetDescr(pgsd), values, nulls); + pgstdat =3D heap_copy_tuple_as_datum(pgstup, RelationGetDescr(pgsd)); + astate =3D accumArrayResult(astate, pgstdat, false, pgstypoid, + CurrentMemoryContext); + + offset +=3D NUM_ATTRIBUTE_STATS_ELEMS; + } + + pfree(exprs_elems); + pfree(exprs_nulls); + + return makeArrayResult(astate, CurrentMemoryContext); +} + +static bool +text_to_float4(Datum input, Datum *output) +{ + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + char *s; + bool ok; + + s =3D TextDatumGetCString(input); + ok =3D DirectInputFunctionCallSafe(float4in, s, InvalidOid, -1, + (Node *) &escontext, output); + + pfree(s); + return ok; +} + + +static bool +text_to_int4(Datum input, Datum *output) +{ + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + char *s; + bool ok; + + s =3D TextDatumGetCString(input); + ok =3D DirectInputFunctionCallSafe(int4in, s, InvalidOid, -1, + (Node *) &escontext, output); + + pfree(s); + return ok; +} + +static bool +delete_pg_statistic_ext_data(Oid stxoid, bool inherited) +{ + Relation sed =3D table_open(StatisticExtDataRelationId, RowExclusiveLock); + HeapTuple oldtup; + bool result =3D false; + + /* Is there already a pg_statistic tuple for this attribute? */ + oldtup =3D SearchSysCache2(STATEXTDATASTXOID, + ObjectIdGetDatum(stxoid), + BoolGetDatum(inherited)); + + if (HeapTupleIsValid(oldtup)) + { + CatalogTupleDelete(sed, &oldtup->t_self); + ReleaseSysCache(oldtup); + result =3D true; + } + + table_close(sed, RowExclusiveLock); + + CommandCounterIncrement(); + + return result; +} + +Datum +pg_restore_extended_stats(PG_FUNCTION_ARGS) +{ + LOCAL_FCINFO(positional_fcinfo, NUM_EXTENDED_STATS_ARGS); + bool result =3D true; + + InitFunctionCallInfoData(*positional_fcinfo, NULL, NUM_EXTENDED_STATS_ARG= S, + InvalidOid, NULL, NULL); + + if (!stats_fill_fcinfo_from_arg_pairs(fcinfo, positional_fcinfo, extargin= fo)) + result =3D false; + + if (!extended_statistics_update(positional_fcinfo)) + result =3D false; + + PG_RETURN_BOOL(result); +} + +/* + * Delete statistics for the given statistics object. + */ +Datum +pg_clear_extended_stats(PG_FUNCTION_ARGS) +{ + char *nspname; + Oid nspoid; + char *stxname; + bool inherited; + Relation pg_stext; + HeapTuple tup; + Relation rel; + Oid locked_table =3D InvalidOid; + + Form_pg_statistic_ext stxform; + + stats_check_required_arg(fcinfo, extarginfo, STATSCHEMA_ARG); + nspname =3D TextDatumGetCString(PG_GETARG_DATUM(STATSCHEMA_ARG)); + stats_check_required_arg(fcinfo, extarginfo, STATNAME_ARG); + stxname =3D TextDatumGetCString(PG_GETARG_DATUM(STATNAME_ARG)); + stats_check_required_arg(fcinfo, extarginfo, INHERITED_ARG); + inherited =3D PG_GETARG_NAME(INHERITED_ARG); + + if (RecoveryInProgress()) + { + ereport(WARNING, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("recovery is in progress"), + errhint("Statistics cannot be modified during recovery."))); + PG_RETURN_VOID(); + } + + nspoid =3D get_namespace_oid(nspname, true); + if (nspoid =3D=3D InvalidOid) + { + ereport(WARNING, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Namespace \"%s\" not found.", stxname))); + PG_RETURN_VOID(); + } + + pg_stext =3D table_open(StatisticExtRelationId, RowExclusiveLock); + tup =3D get_pg_statistic_ext(pg_stext, nspoid, stxname); + + if (!HeapTupleIsValid(tup)) + { + table_close(pg_stext, RowExclusiveLock); + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("Extended Statistics Object \"%s\".\"%s\" not found.", + nspname, stxname))); + PG_RETURN_VOID(); + } + + stxform =3D (Form_pg_statistic_ext) GETSTRUCT(tup); + + /* Roundabout way of getting a RangeVar on the underlying table */ + rel =3D relation_open(stxform->stxrelid, AccessShareLock); + + /* no need to fetch reloid, we already have it */ + RangeVarGetRelidExtended(makeRangeVar(nspname, + RelationGetRelationName(rel), -1), + ShareUpdateExclusiveLock, 0, + RangeVarCallbackForStats, &locked_table); + + relation_close(rel, AccessShareLock); + + delete_pg_statistic_ext_data(stxform->oid, inherited); + heap_freetuple(tup); + table_close(pg_stext, RowExclusiveLock); + + PG_RETURN_VOID(); +} diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c index f59fb8215437..a917079ceb0b 100644 --- a/src/backend/statistics/mcv.c +++ b/src/backend/statistics/mcv.c @@ -2173,3 +2173,147 @@ mcv_clause_selectivity_or(PlannerInfo *root, Statis= ticExtInfo *stat, =20 return s; } + +/* + * The MCV is an array of records, but this is expected as 4 separate arr= ays. + * It is not possible to have a generic input function for pg_mcv_list + * because the most_common_values is a composite type with element types + * defined by the specific statistics object. + */ +Datum +import_mcvlist(HeapTuple tup, int elevel, int numattrs, Oid *atttypids, + int32 *atttypmods, Oid *atttypcolls, int nitems, + Datum *mcv_elems, bool *mcv_nulls, + bool *mcv_elem_nulls, float8 *freqs, float8 *base_freqs) +{ + MCVList *mcvlist; + bytea *bytes; + + HeapTuple *vatuples; + VacAttrStats **vastats; + + /* + * Allocate the MCV list structure, set the global parameters. + */ + mcvlist =3D (MCVList *) palloc0(offsetof(MCVList, items) + + (sizeof(MCVItem) * nitems)); + + mcvlist->magic =3D STATS_MCV_MAGIC; + mcvlist->type =3D STATS_MCV_TYPE_BASIC; + mcvlist->ndimensions =3D numattrs; + mcvlist->nitems =3D nitems; + + /* Set the values for the 1-D arrays and allocate space for the 2-D array= s */ + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + item->frequency =3D freqs[i]; + item->base_frequency =3D base_freqs[i]; + item->values =3D (Datum *) palloc0(sizeof(Datum) * numattrs); + item->isnull =3D (bool *) palloc0(sizeof(bool) * numattrs); + } + + /* Walk through each dimension */ + for (int j =3D 0; j < numattrs; j++) + { + FmgrInfo finfo; + Oid ioparam; + Oid infunc; + int index =3D j; + + getTypeInputInfo(atttypids[j], &infunc, &ioparam); + fmgr_info(infunc, &finfo); + + /* store info about data type OIDs */ + mcvlist->types[j] =3D atttypids[j]; + + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + /* These should be in agreement, but just to be safe check both */ + if (mcv_elem_nulls[index] || mcv_nulls[index]) + { + item->values[j] =3D (Datum) 0; + item->isnull[j] =3D true; + } + else + { + char *s =3D TextDatumGetCString(mcv_elems[index]); + ErrorSaveContext escontext =3D {T_ErrorSaveContext}; + + if (!InputFunctionCallSafe(&finfo, s, ioparam, atttypmods[j], + (Node *) &escontext, &item->values[j])) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("MCV elemement \"%s\" does not match expected input type.",= s))); + return (Datum) 0; + } + + pfree(s); + } + + index +=3D numattrs; + } + } + + /* + * The function statext_mcv_serialize() requires an array of pointers to + * VacAttrStats records, but only a few fields within those records have + * to be filled out. + */ + vastats =3D (VacAttrStats **) palloc0(numattrs * sizeof(VacAttrStats)); + vatuples =3D (HeapTuple *) palloc0(numattrs * sizeof(HeapTuple)); + + for (int i =3D 0; i < numattrs; i++) + { + Oid typid =3D atttypids[i]; + HeapTuple typtuple; + + typtuple =3D SearchSysCacheCopy1(TYPEOID, ObjectIdGetDatum(typid)); + + if (!HeapTupleIsValid(typtuple)) + elog(ERROR, "cache lookup failed for type %u", typid); + + vatuples[i] =3D typtuple; + + vastats[i] =3D palloc0(sizeof(VacAttrStats)); + + vastats[i]->attrtype =3D (Form_pg_type) GETSTRUCT(typtuple); + vastats[i]->attrtypid =3D typid; + vastats[i]->attrcollid =3D atttypcolls[i]; + } + + bytes =3D statext_mcv_serialize(mcvlist, vastats); + + for (int i =3D 0; i < numattrs; i++) + { + pfree(vatuples[i]); + pfree(vastats[i]); + } + pfree((void *) vatuples); + pfree((void *) vastats); + + if (bytes =3D=3D NULL) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("Unable to import mcv list"))); + return (Datum) 0; + } + + for (int i =3D 0; i < nitems; i++) + { + MCVItem *item =3D &mcvlist->items[i]; + + pfree(item->values); + pfree(item->isnull); + } + pfree(mcvlist); + pfree(mcv_elems); + pfree(mcv_nulls); + + return PointerGetDatum(bytes); +} diff --git a/src/backend/statistics/mvdistinct.c b/src/backend/statistics/m= vdistinct.c index fe452f53ae4b..839bcc9af929 100644 --- a/src/backend/statistics/mvdistinct.c +++ b/src/backend/statistics/mvdistinct.c @@ -599,6 +599,68 @@ generate_combinations_recurse(CombinationGenerator *st= ate, } } =20 +/* + * Free allocations of an MVNDistinct + */ +void +free_pg_ndistinct(MVNDistinct *ndistinct) +{ + for (int i =3D 0; i < ndistinct->nitems; i++) + pfree(ndistinct->items[i].attributes); + + pfree(ndistinct); +} + +/* + * Validate an MVNDistinct against the extended statistics object definiti= on. + * + * Every MVNDistinctItem must be checked to ensure that the attnums in the + * attributes list correspond to attnums/expressions defined by the + * extended statistics object. + * + * Positive attnums are attributes which must be found in the stxkeys, + * while negative attnums correspond to an expr number, so the attnum + * can't be below (0 - numexprs). + */ +bool +pg_ndistinct_validate_items(MVNDistinct *ndistinct, int2vector *stxkeys, i= nt numexprs, int elevel) +{ + int attnum_expr_lowbound =3D 0 - numexprs; + + for (int i =3D 0; i < ndistinct->nitems; i++) + { + MVNDistinctItem item =3D ndistinct->items[i]; + + for (int j =3D 0; j < item.nattributes; j++) + { + AttrNumber attnum =3D item.attributes[j]; + bool ok =3D false; + + if (attnum > 0) + { + for (int k =3D 0; k < stxkeys->dim1; k++) + if (attnum =3D=3D stxkeys->values[k]) + { + ok =3D true; + break; + } + } + else if ((attnum < 0) && (attnum >=3D attnum_expr_lowbound)) + ok =3D true; + + if (!ok) + { + ereport(elevel, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("pg_ndistinct: invalid attnum for this statistics object: %d"= , attnum))); + return false; + } + } + } + return true; +} + + /* * generate_combinations * generate all k-combinations of N elements diff --git a/src/test/regress/expected/stats_import.out b/src/test/regress/= expected/stats_import.out index 98ce7dc28410..970e9bd09833 100644 --- a/src/test/regress/expected/stats_import.out +++ b/src/test/regress/expected/stats_import.out @@ -1084,11 +1084,15 @@ SELECT 3, 'tre', (3, 3.3, 'TRE', '2003-03-03', NULL= )::stats_import.complex_type, UNION ALL SELECT 4, 'four', NULL, int4range(0,100), NULL; CREATE INDEX is_odd ON stats_import.test(((comp).a % 2 =3D 1)); +CREATE STATISTICS stats_import.test_stat ON name, comp, lower(arange), arr= ay_length(tags,1) +FROM stats_import.test; -- Generate statistics on table with data ANALYZE stats_import.test; CREATE TABLE stats_import.test_clone ( LIKE stats_import.test ) WITH (autovacuum_enabled =3D false); CREATE INDEX is_odd_clone ON stats_import.test_clone(((comp).a % 2 =3D 1)); +CREATE STATISTICS stats_import.test_stat_clone ON name, comp, lower(arange= ), array_length(tags,1) +FROM stats_import.test_clone; -- -- Copy stats from test to test_clone, and is_odd to is_odd_clone -- @@ -1342,6 +1346,1125 @@ AND attname =3D 'i'; (1 row) =20 DROP TABLE stats_temp; +-- set n_distinct using at attnum (1) that is not in the statistics object +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2,1], "ndistinct" : 4}]':= :pg_ndistinct + ); +WARNING: pg_ndistinct: invalid attnum for this statistics object: 1 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set n_distinct using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2,0], "ndistinct" : 4}]':= :pg_ndistinct + ); +WARNING: pg_ndistinct: invalid attnum for this statistics object: 0 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set n_distinct using at attnum that is outside the expression bounds(be= low -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [-4,3,-1], "ndistinct" : 4}, + {"attributes" : [-4,3,-2], "ndistinct" : 4}, + {"attributes" : [-4,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2,-4], "ndistinct" : 4}]'::= pg_ndistinct + ); +WARNING: pg_ndistinct: invalid attnum for this statistics object: -4 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2], "ndistinct" : 4}]'::p= g_ndistinct + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + e.dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+------------------------ +n_distinct | [ + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | 3 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 3, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 3, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 3,+ + | "attributes": [+ + | -1, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | 3, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | 3, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | -1, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4,+ + | "attributes": [+ + | 2, + + | 3, + + | -1, + + | -2 + + | ] + + | } + + | ] +dependencies |=20 +most_common_vals |=20 +most_common_val_nulls |=20 +most_common_freqs |=20 +most_common_base_freqs |=20 + +-- set dependencies using at attnum (1) that is not in the statistics obje= ct +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 1, "degree": 1= =2E000000}]'::pg_dependencies + ); +WARNING: pg_dependencies: invalid attnum for this statistics object: 1 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set dependencies using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [0], "dependency": -1, "degree": = 1.000000}]'::pg_dependencies + ); +WARNING: pg_dependencies: invalid attnum for this statistics object: 0 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- set dependencies using at attnum that is outside the expression bounds(= below -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": -3, "degree": = 1.000000}]'::pg_dependencies + ); +WARNING: pg_dependencies: invalid attnum for this statistics object: -3 + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, + {"attributes": [2], "dependency": -1, "degree": = 1.000000}, + {"attributes": [2], "dependency": -2, "degree": = 1.000000}, + {"attributes": [3], "dependency": 2, "degree": 1= =2E000000}, + {"attributes": [3], "dependency": -1, "degree": = 1.000000}, + {"attributes": [3], "dependency": -2, "degree": = 1.000000}, + {"attributes": [-1], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-1], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-1], "dependency": -2, "degree":= 1.000000}, + {"attributes": [-2], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-2], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-2], "dependency": -1, "degree":= 1.000000}, + {"attributes": [2,3], "dependency": -1, "degree"= : 1.000000}, + {"attributes": [2,3], "dependency": -2, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [2,-2], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [3,-1], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [3,-2], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [-1,-2], "dependency": 2, "degree= ": 0.500000}, + {"attributes": [-1,-2], "dependency": 3, "degree= ": 0.500000}, + {"attributes": [2,3,-2], "dependency": -1, "degr= ee": 1.000000}, + {"attributes": [2,-1,-2], "dependency": 3, "degr= ee": 1.000000}, + {"attributes": [3,-1,-2], "dependency": 2, "degr= ee": 1.000000}]'::pg_dependencies + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + jsonb_pretty(e.dependencies::text::jsonb) AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+---------------------------- +n_distinct | [ + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | 3 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 3, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 3, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 3, + + | "attributes": [ + + | -1, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | 3, + + | -1 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | 3, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | -1, + + | -2 + + | ] + + | }, + + | { + + | "ndistinct": 4, + + | "attributes": [ + + | 2, + + | 3, + + | -1, + + | -2 + + | ] + + | } + + | ] +dependencies | [ + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -1 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -1 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | -1 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -2 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -2 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | -2 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | 3 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | 3 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | -1 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | -1 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | -2 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | -2 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3, + + | -1 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3, + + | -1 + + | ], + + | "dependency": -2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3, + + | -2 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3, + + | -2 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -1, + + | -2 + + | ], + + | "dependency": 2 + + | }, + + | { + + | "degree": 0.500000,+ + | "attributes": [ + + | -1, + + | -2 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | 3, + + | -2 + + | ], + + | "dependency": -1 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 2, + + | -1, + + | -2 + + | ], + + | "dependency": 3 + + | }, + + | { + + | "degree": 1.000000,+ + | "attributes": [ + + | 3, + + | -1, + + | -2 + + | ], + + | "dependency": 2 + + | } + + | ] +most_common_vals |=20 +most_common_val_nulls |=20 +most_common_freqs |=20 +most_common_base_freqs |=20 + +-- if any one mcv param specified, all four must be specified (part 1) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 3) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- if any one mcv param specified, all four must be specified (part 4) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); +WARNING: MCV parameters "most_common_vals", "most_common_val_nulls", "mos= t_common_freqs", and "most_common_base_freqs" must be all specified if any = are specified + pg_restore_extended_stats=20 +--------------------------- + f +(1 row) + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[= ], + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[], + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[], + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + jsonb_pretty(e.dependencies::text::jsonb) AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx +-[ RECORD 1 ]----------+--------------------------------------------------= ---------------------------------------------------------------------------= --------------------------------------------------------------- +n_distinct | [ = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | 3 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | -1 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 3, = = + + | -1 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 3, = = + + | -2 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 3, = = + + | "attributes": [ = = + + | -1, = = + + | -2 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | 3, = = + + | -1 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | 3, = = + + | -2 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | -1, = = + + | -2 = = + + | ] = = + + | }, = = + + | { = = + + | "ndistinct": 4, = = + + | "attributes": [ = = + + | 2, = = + + | 3, = = + + | -1, = = + + | -2 = = + + | ] = = + + | } = = + + | ] +dependencies | [ = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -1 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -1 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | -1 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -2 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -2 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | -2 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | 3 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | 3 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | -1 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | -1 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | -2 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | -2 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3, = = + + | -1 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3, = = + + | -1 = = + + | ], = = + + | "dependency": -2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3, = = + + | -2 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3, = = + + | -2 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -1, = = + + | -2 = = + + | ], = = + + | "dependency": 2 = = + + | }, = = + + | { = = + + | "degree": 0.500000, = = + + | "attributes": [ = = + + | -1, = = + + | -2 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | 3, = = + + | -2 = = + + | ], = = + + | "dependency": -1 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 2, = = + + | -1, = = + + | -2 = = + + | ], = = + + | "dependency": 3 = = + + | }, = = + + | { = = + + | "degree": 1.000000, = = + + | "attributes": [ = = + + | 3, = = + + | -1, = = + + | -2 = = + + | ], = = + + | "dependency": 2 = = + + | } = = + + | ] +most_common_vals | {{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-2001,\= "{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-1,3}= ,{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}} +most_common_val_nulls | {{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}} +most_common_freqs | {0.25,0.25,0.25,0.25} +most_common_base_freqs | {0.00390625,0.015625,0.00390625,0.015625} + +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL,NULL,NULL},= {0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[] + ); + pg_restore_extended_stats=20 +--------------------------- + t +(1 row) + +SELECT + e.inherited, e.null_frac, e.avg_width, e.n_distinct, e.most_common_val= s, + e.most_common_freqs, e.histogram_bounds, e.correlation, + e.most_common_elems, e.most_common_elem_freqs, e.elem_count_histogram +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +and e.inherited =3D false +\gx +-[ RECORD 1 ]----------+------- +inherited | f +null_frac | 0 +avg_width | 4 +n_distinct | -0.75 +most_common_vals | {1} +most_common_freqs | {0.5} +histogram_bounds | {-1,0} +correlation | -0.6 +most_common_elems |=20 +most_common_elem_freqs |=20 +elem_count_histogram |=20 +-[ RECORD 2 ]----------+------- +inherited | f +null_frac | 0.25 +avg_width | 4 +n_distinct | -0.5 +most_common_vals | {2} +most_common_freqs | {0.5} +histogram_bounds |=20 +correlation | 1 +most_common_elems |=20 +most_common_elem_freqs |=20 +elem_count_histogram |=20 + +SELECT + pg_catalog.pg_clear_extended_stats( + statistics_schemaname =3D> 'stats_import', + statistics_name =3D> 'test_stat_clone', + inherited =3D> false); + pg_clear_extended_stats=20 +------------------------- +=20 +(1 row) + +SELECT COUNT(*) +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + count=20 +------- + 0 +(1 row) + +SELECT COUNT(*) +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + count=20 +------- + 0 +(1 row) + +-- +-- Copy stats from test_stat to test_stat_clone +-- +SELECT + e.statistics_name, + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', e.statistics_schemaname::text, + 'statistics_name', 'test_stat_clone', + 'inherited', e.inherited, + 'n_distinct', e.n_distinct, + 'dependencies', e.dependencies, + 'most_common_vals', e.most_common_vals, + 'most_common_val_nulls', e.most_common_val_nulls, + 'most_common_freqs', e.most_common_freqs, + 'most_common_base_freqs', e.most_common_base_freqs, + 'exprs', x.exprs + ) +FROM pg_stats_ext AS e +CROSS JOIN LATERAL ( + SELECT + array_agg( + ARRAY[ee.null_frac::text, ee.avg_width::text, + ee.n_distinct::text, ee.most_common_vals::text, + ee.most_common_freqs::text, ee.histogram_bounds::text, + ee.correlation::text, ee.most_common_elems::text, + ee.most_common_elem_freqs::text, + ee.elem_count_histogram::text]) + FROM pg_stats_ext_exprs AS ee + WHERE ee.statistics_schemaname =3D e.statistics_schemaname + AND ee.statistics_name =3D e.statistics_name + AND ee.inherited =3D e.inherited + ) AS x(exprs) +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat'; + statistics_name | pg_restore_extended_stats=20 +-----------------+--------------------------- + test_stat | t +(1 row) + +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + inherited | n_distinct | dependencies | most_common_vals | most_common_va= l_nulls | most_common_freqs | most_common_base_freqs=20 +-----------+------------+--------------+------------------+---------------= --------+-------------------+------------------------ +(0 rows) + +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + inherited | n_distinct | dependencies | most_common_vals | most_common_va= l_nulls | most_common_freqs | most_common_base_freqs=20 +-----------+------------+--------------+------------------+---------------= --------+-------------------+------------------------ +(0 rows) + +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + inherited | null_frac | avg_width | n_distinct | most_common_vals | most_= common_freqs | histogram_bounds | correlation | most_common_elems | most_co= mmon_elem_freqs | elem_count_histogram=20 +-----------+-----------+-----------+------------+------------------+------= -------------+------------------+-------------+-------------------+--------= ----------------+---------------------- +(0 rows) + +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + inherited | null_frac | avg_width | n_distinct | most_common_vals | most_= common_freqs | histogram_bounds | correlation | most_common_elems | most_co= mmon_elem_freqs | elem_count_histogram=20 +-----------+-----------+-----------+------------+------------------+------= -------------+------------------+-------------+-------------------+--------= ----------------+---------------------- +(0 rows) + DROP SCHEMA stats_import CASCADE; NOTICE: drop cascades to 6 other objects DETAIL: drop cascades to type stats_import.complex_type diff --git a/src/test/regress/sql/stats_import.sql b/src/test/regress/sql/s= tats_import.sql index d140733a7502..48a03f5b8031 100644 --- a/src/test/regress/sql/stats_import.sql +++ b/src/test/regress/sql/stats_import.sql @@ -766,6 +766,9 @@ SELECT 4, 'four', NULL, int4range(0,100), NULL; =20 CREATE INDEX is_odd ON stats_import.test(((comp).a % 2 =3D 1)); =20 +CREATE STATISTICS stats_import.test_stat ON name, comp, lower(arange), arr= ay_length(tags,1) +FROM stats_import.test; + -- Generate statistics on table with data ANALYZE stats_import.test; =20 @@ -774,6 +777,9 @@ CREATE TABLE stats_import.test_clone ( LIKE stats_impor= t.test ) =20 CREATE INDEX is_odd_clone ON stats_import.test_clone(((comp).a % 2 =3D 1)); =20 +CREATE STATISTICS stats_import.test_stat_clone ON name, comp, lower(arange= ), array_length(tags,1) +FROM stats_import.test_clone; + -- -- Copy stats from test to test_clone, and is_odd to is_odd_clone -- @@ -970,4 +976,362 @@ AND tablename =3D 'stats_temp' AND inherited =3D false AND attname =3D 'i'; DROP TABLE stats_temp; + +-- set n_distinct using at attnum (1) that is not in the statistics object +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2,1], "ndistinct" : 4}]':= :pg_ndistinct + ); + +-- set n_distinct using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2,0], "ndistinct" : 4}]':= :pg_ndistinct + ); + +-- set n_distinct using at attnum that is outside the expression bounds(be= low -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [-4,3,-1], "ndistinct" : 4}, + {"attributes" : [-4,3,-2], "ndistinct" : 4}, + {"attributes" : [-4,-1,-2], "ndistinct" : 4}, + {"attributes" : [3,-1,-2,-4], "ndistinct" : 4}]'::= pg_ndistinct + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'n_distinct', '[{"attributes" : [2,3], "ndistinct" : 4}, + {"attributes" : [2,-1], "ndistinct" : 4}, + {"attributes" : [3,-1], "ndistinct" : 4}, + {"attributes" : [3,-2], "ndistinct" : 4}, + {"attributes" : [-1,-2], "ndistinct" : 3}, + {"attributes" : [2,3,-1], "ndistinct" : 4}, + {"attributes" : [2,3,-2], "ndistinct" : 4}, + {"attributes" : [2,-1,-2], "ndistinct" : 4}, + {"attributes" : [2,3,-1,-2], "ndistinct" : 4}]'::p= g_ndistinct + ); + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + e.dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +-- set dependencies using at attnum (1) that is not in the statistics obje= ct +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 1, "degree": 1= =2E000000}]'::pg_dependencies + ); + +-- set dependencies using at attnum that is 0 +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [0], "dependency": -1, "degree": = 1.000000}]'::pg_dependencies + ); + +-- set dependencies using at attnum that is outside the expression bounds(= below -2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": -3, "degree": = 1.000000}]'::pg_dependencies + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'dependencies', '[{"attributes": [2], "dependency": 3, "degree": 1= =2E000000}, + {"attributes": [2], "dependency": -1, "degree": = 1.000000}, + {"attributes": [2], "dependency": -2, "degree": = 1.000000}, + {"attributes": [3], "dependency": 2, "degree": 1= =2E000000}, + {"attributes": [3], "dependency": -1, "degree": = 1.000000}, + {"attributes": [3], "dependency": -2, "degree": = 1.000000}, + {"attributes": [-1], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-1], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-1], "dependency": -2, "degree":= 1.000000}, + {"attributes": [-2], "dependency": 2, "degree": = 0.500000}, + {"attributes": [-2], "dependency": 3, "degree": = 0.500000}, + {"attributes": [-2], "dependency": -1, "degree":= 1.000000}, + {"attributes": [2,3], "dependency": -1, "degree"= : 1.000000}, + {"attributes": [2,3], "dependency": -2, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [2,-2], "dependency": 3, "degree"= : 1.000000}, + {"attributes": [2,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [3,-1], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-1], "dependency": -2, "degree= ": 1.000000}, + {"attributes": [3,-2], "dependency": 2, "degree"= : 1.000000}, + {"attributes": [3,-2], "dependency": -1, "degree= ": 1.000000}, + {"attributes": [-1,-2], "dependency": 2, "degree= ": 0.500000}, + {"attributes": [-1,-2], "dependency": 3, "degree= ": 0.500000}, + {"attributes": [2,3,-2], "dependency": -1, "degr= ee": 1.000000}, + {"attributes": [2,-1,-2], "dependency": 3, "degr= ee": 1.000000}, + {"attributes": [3,-1,-2], "dependency": 2, "degr= ee": 1.000000}]'::pg_dependencies + ); + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + jsonb_pretty(e.dependencies::text::jsonb) AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +-- if any one mcv param specified, all four must be specified (part 1) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[] + ); + +-- if any one mcv param specified, all four must be specified (part 2) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[] + ); + +-- if any one mcv param specified, all four must be specified (part 3) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[] + ); + +-- if any one mcv param specified, all four must be specified (part 4) +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + +-- ok +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'most_common_vals', '{{four,NULL,0,NULL},{one,"(1,1.1,ONE,01-01-20= 01,\"{\"\"xkey\"\": \"\"xval\"\"}\")",1,2},{tre,"(3,3.3,TRE,03-03-2003,)",-= 1,3},{two,"(2,2.2,TWO,02-02-2002,\"[true, 4, \"\"six\"\"]\")",1,2}}'::text[= ], + 'most_common_val_nulls', '{{f,t,f,t},{f,f,f,f},{f,f,f,f},{f,f,f,f}= }'::boolean[], + 'most_common_freqs', '{0.25,0.25,0.25,0.25}'::double precision[], + 'most_common_base_freqs', '{0.00390625,0.015625,0.00390625,0.01562= 5}'::double precision[] + ); + +SELECT + jsonb_pretty(e.n_distinct::text::jsonb) AS n_distinct, + jsonb_pretty(e.dependencies::text::jsonb) AS dependencies, + e.most_common_vals, e.most_common_val_nulls, + e.most_common_freqs, e.most_common_base_freqs +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false +\gx + +SELECT + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', 'stats_import', + 'statistics_name', 'test_stat_clone', + 'inherited', false, + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL,NULL,NULL},= {0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[] + ); + +SELECT + e.inherited, e.null_frac, e.avg_width, e.n_distinct, e.most_common_val= s, + e.most_common_freqs, e.histogram_bounds, e.correlation, + e.most_common_elems, e.most_common_elem_freqs, e.elem_count_histogram +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +and e.inherited =3D false +\gx + +SELECT + pg_catalog.pg_clear_extended_stats( + statistics_schemaname =3D> 'stats_import', + statistics_name =3D> 'test_stat_clone', + inherited =3D> false); + +SELECT COUNT(*) +FROM pg_stats_ext AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + +SELECT COUNT(*) +FROM pg_stats_ext_exprs AS e +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat_clone' +AND e.inherited =3D false; + +-- +-- Copy stats from test_stat to test_stat_clone +-- +SELECT + e.statistics_name, + pg_catalog.pg_restore_extended_stats( + 'statistics_schemaname', e.statistics_schemaname::text, + 'statistics_name', 'test_stat_clone', + 'inherited', e.inherited, + 'n_distinct', e.n_distinct, + 'dependencies', e.dependencies, + 'most_common_vals', e.most_common_vals, + 'most_common_val_nulls', e.most_common_val_nulls, + 'most_common_freqs', e.most_common_freqs, + 'most_common_base_freqs', e.most_common_base_freqs, + 'exprs', x.exprs + ) +FROM pg_stats_ext AS e +CROSS JOIN LATERAL ( + SELECT + array_agg( + ARRAY[ee.null_frac::text, ee.avg_width::text, + ee.n_distinct::text, ee.most_common_vals::text, + ee.most_common_freqs::text, ee.histogram_bounds::text, + ee.correlation::text, ee.most_common_elems::text, + ee.most_common_elem_freqs::text, + ee.elem_count_histogram::text]) + FROM pg_stats_ext_exprs AS ee + WHERE ee.statistics_schemaname =3D e.statistics_schemaname + AND ee.statistics_name =3D e.statistics_name + AND ee.inherited =3D e.inherited + ) AS x(exprs) +WHERE e.statistics_schemaname =3D 'stats_import' +AND e.statistics_name =3D 'test_stat'; + +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + +SELECT n.inherited, + n.n_distinct, n.dependencies, n.most_common_vals, + n.most_common_val_nulls, n.most_common_freqs, + n.most_common_base_freqs +FROM pg_stats_ext AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.n_distinct, o.dependencies, o.most_common_vals, + o.most_common_val_nulls, o.most_common_freqs, + o.most_common_base_freqs +FROM pg_stats_ext AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat' +EXCEPT +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone'; + +SELECT n.inherited, + n.null_frac, n.avg_width, n.n_distinct, + n.most_common_vals::text AS most_common_vals, + n.most_common_freqs, + n.histogram_bounds::text AS histogram_bounds, + n.correlation, + n.most_common_elems::text AS most_common_elems, + n.most_common_elem_freqs, n.elem_count_histogram +FROM pg_stats_ext_exprs AS n +WHERE n.statistics_schemaname =3D 'stats_import' +AND n.statistics_name =3D 'test_stat_clone' +EXCEPT +SELECT o.inherited, + o.null_frac, o.avg_width, o.n_distinct, + o.most_common_vals::text AS most_common_vals, + o.most_common_freqs, + o.histogram_bounds::text AS histogram_bounds, + o.correlation, + o.most_common_elems::text AS most_common_elems, + o.most_common_elem_freqs, o.elem_count_histogram +FROM pg_stats_ext_exprs AS o +WHERE o.statistics_schemaname =3D 'stats_import' +AND o.statistics_name =3D 'test_stat'; + DROP SCHEMA stats_import CASCADE; diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-adm= in.sgml index 1b465bc8ba71..574d4a35a64f 100644 --- a/doc/src/sgml/func/func-admin.sgml +++ b/doc/src/sgml/func/func-admin.sgml @@ -2167,6 +2167,104 @@ SELECT pg_restore_attribute_stats( + + + + pg_restore_extended_stats + + pg_restore_extended_stats ( + VARIADIC kwargs "a= ny" ) + boolean + + + Creates or updates statistics for statistics objects. Ordinarily, + these statistics are collected automatically or updated as a part= of + or = , so + it's not necessary to call this function. However, it is useful + after a restore to enable the optimizer to choose better plans if + ANALYZE has not been run yet. + + + The tracked statistics may change from version to version, so + arguments are passed as pairs of argname + and argvalue in the form: + + SELECT pg_restore_extended_stats( + 'arg1name', 'arg1value'::arg1type, + 'arg2name', 'arg2value'::arg2type, + 'arg3name', 'arg3value'::arg3type); + + + + For example, to set the n_distinct, + dependencies, and exprs + values for the statistics object myschema.mystatsobj<= /structname>: + + SELECT pg_restore_extended_stats( + 'statistics_schemaname', 'myschema'::name, + 'statistics_name', 'mytable'::name, + 'inherited', false, + 'n_distinct', '{"2, 3": 4, "2, -1": 4, "2, -2": 4, "3, -1":= 4, "3, -2": 4}'::pg_ndistinct, + 'dependencies', '{"2 =3D> 1": 1.000000, "2 =3D> -1": 1.000000= , "2 =3D> -2": 1.000000}'::pg_dependencies + 'exprs', '{{0,4,-0.75,"{1}","{0.5}","{-1,0}",-0.6,NULL= ,NULL,NULL},{0.25,4,-0.5,"{2}","{0.5}",NULL,1,NULL,NULL,NULL}}'::text[]); + + + + The required arguments are statistics_schemaname with a value + of type name, which specifies the statistics object'= s schema; + statistics_name with a value of type nam= e, which specifies + the name of the statistics object; and inherited, which + specifies whether the statistics include values from child tables. + Other arguments are the names and values of statistics correspond= ing + to columns in pg_= stats_ext + . To accept statistics for any expressions in the extended= statistics object, the + parameter exprs with a type text[] is available, the array + must be two dimensional with an outer array in length equal to th= e number of expressions in + the object, and the inner array elements for each of the statisti= cal columns in pg_stats_ext_expr= s, some + of which are themselves arrays. + + + Additionally, this function accepts argument name + version of type integer, which + specifies the server version from which the statistics originated. + This is anticipated to be helpful in porting statistics from older + versions of PostgreSQL. + + + Minor errors are reported as a WARNING and + ignored, and remaining statistics will still be restored. If all + specified statistics are successfully restored, returns + true, otherwise false. + + + The caller must have the MAINTAIN privilege on= the + table or be the owner of the database. + + + + + + + + pg_clear_extended_stats + + pg_clear_extended_stats ( + statistics_schemaname name, + statistics_name name, + inherited boolean ) + void + + + Clears statistics for the given statistics object, as + though the object was newly created. + + + The caller must have the MAINTAIN privilege on + the table or be the owner of the database. + + + --=20 2.51.0 --AorqrXjCfzfo1Gz1 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v14-0005-Include-Extended-Statistics-in-pg_dump.patch" Content-Transfer-Encoding: quoted-printable =46rom 8930510e729fff8c99cae4ea50de1710583f0dfa Mon Sep 17 00:00:00 2001 =46rom: Corey Huinker Date: Wed, 5 Nov 2025 01:13:04 -0500 Subject: [PATCH v14 5/5] Include Extended Statistics in pg_dump. Incorporate the new pg_restore_extended_stats() function into pg_dump. This detects the existence of extended statistics statistics (i.e. pg_statistic_ext_data rows). This handles many of the changes that have happened to extended statistic statistics over the various versions, including: * Format change for pg_ndistinct and pg_dependencies in current development version. Earlier versions have the format translated via the pg_dump SQL statement. * Inherited extended statistics were introduced in v15. * Expressions were introduced to extended statistics in v14. * MCV extended statistics were introduced in v13. * pg_statistic_ext_data and pg_stats_ext introduced in v12, prior to that ndstinct and depdendencies data (the only kind of stats that existed were directly on pg_statistic_ext. * Extended Statistics were introduced in v10, so there is no support for prior versions necessary. --- src/bin/pg_dump/pg_backup.h | 1 + src/bin/pg_dump/pg_backup_archiver.c | 3 +- src/bin/pg_dump/pg_dump.c | 252 +++++++++++++++++++++++++++ src/bin/pg_dump/t/002_pg_dump.pl | 28 +++ 4 files changed, 283 insertions(+), 1 deletion(-) diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h index d9041dad7206..df708e4ced69 100644 --- a/src/bin/pg_dump/pg_backup.h +++ b/src/bin/pg_dump/pg_backup.h @@ -68,6 +68,7 @@ enum _dumpPreparedQueries PREPQUERY_DUMPCOMPOSITETYPE, PREPQUERY_DUMPDOMAIN, PREPQUERY_DUMPENUMTYPE, + PREPQUERY_DUMPEXTSTATSSTATS, PREPQUERY_DUMPFUNC, PREPQUERY_DUMPOPR, PREPQUERY_DUMPRANGETYPE, diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_back= up_archiver.c index c84b017f21b8..ab9cb75a86f8 100644 --- a/src/bin/pg_dump/pg_backup_archiver.c +++ b/src/bin/pg_dump/pg_backup_archiver.c @@ -3008,7 +3008,8 @@ _tocEntryRequired(TocEntry *te, teSection curSection,= ArchiveHandle *AH) strcmp(te->desc, "SEARCHPATH") =3D=3D 0) return REQ_SPECIAL; =20 - if (strcmp(te->desc, "STATISTICS DATA") =3D=3D 0) + if ((strcmp(te->desc, "STATISTICS DATA") =3D=3D 0) || + (strcmp(te->desc, "EXTENDED STATISTICS DATA") =3D=3D 0)) { if (!ropt->dumpStatistics) return 0; diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index a00918bacb40..8c5850f9e9b3 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -324,6 +324,7 @@ static void dumpSequenceData(Archive *fout, const Table= DataInfo *tdinfo); static void dumpIndex(Archive *fout, const IndxInfo *indxinfo); static void dumpIndexAttach(Archive *fout, const IndexAttachInfo *attachin= fo); static void dumpStatisticsExt(Archive *fout, const StatsExtInfo *statsexti= nfo); +static void dumpStatisticsExtStats(Archive *fout, const StatsExtInfo *stat= sextinfo); static void dumpConstraint(Archive *fout, const ConstraintInfo *coninfo); static void dumpTableConstraintComment(Archive *fout, const ConstraintInfo= *coninfo); static void dumpTSParser(Archive *fout, const TSParserInfo *prsinfo); @@ -8258,6 +8259,9 @@ getExtendedStatistics(Archive *fout) =20 /* Decide whether we want to dump it */ selectDumpableStatisticsObject(&(statsextinfo[i]), fout); + + if (fout->dopt->dumpStatistics) + statsextinfo[i].dobj.components |=3D DUMP_COMPONENT_STATISTICS; } =20 PQclear(res); @@ -11712,6 +11716,7 @@ dumpDumpableObject(Archive *fout, DumpableObject *d= obj) break; case DO_STATSEXT: dumpStatisticsExt(fout, (const StatsExtInfo *) dobj); + dumpStatisticsExtStats(fout, (const StatsExtInfo *) dobj); break; case DO_REFRESH_MATVIEW: refreshMatViewData(fout, (const TableDataInfo *) dobj); @@ -18514,6 +18519,253 @@ dumpStatisticsExt(Archive *fout, const StatsExtIn= fo *statsextinfo) free(qstatsextname); } =20 +/* + * dumpStatisticsExtStats + * write out to fout the stats for an extended statistics object + */ +static void +dumpStatisticsExtStats(Archive *fout, const StatsExtInfo *statsextinfo) +{ + DumpOptions *dopt =3D fout->dopt; + PQExpBuffer query; + PGresult *res; + int nstats; + + /* Do nothing if not dumping statistics */ + if (!dopt->dumpStatistics) + return; + + if (!fout->is_prepared[PREPQUERY_DUMPEXTSTATSSTATS]) + { + PQExpBuffer pq =3D createPQExpBuffer(); + + /* + * Set up query for constraint-specific details. + * + * 19+: query pg_stats_ext and pg_stats_ext_exprs as-is 15-18: query + * pg_stats_ext translating the ndistinct and depdendencies, 14: + * inherited is always NULL 12-13: no pg_stats_ext_exprs 10-11: no + * pg_stats_ext, join pg_statistic_ext and pg_namespace + */ + + appendPQExpBufferStr(pq, + "PREPARE getExtStatsStats(pg_catalog.name, pg_catalog.name) AS\n" + "SELECT "); + + /* + * Versions 15+ have inherited stats. + * + * Create this column in all version because we need to order by it late= r. + */ + if (fout->remoteVersion >=3D 150000) + appendPQExpBufferStr(pq, "e.inherited, "); + else + appendPQExpBufferStr(pq, "false AS inherited, "); + + /* + * Versions < 19 use the old ndistintinct and depdendencies formats + * + * These transformations may look scary, but all we're doing is translat= ing + * + * {"3, 4": 11, "3, 6": 11, "4, 6": 11, "3, 4, 6": 11} + * + * to + * + * [{"ndistinct": 11, "attributes": [3,4]}, + * {"ndistinct": 11, "attributes": [3,6]}, + * {"ndistinct": 11, "attributes": [4,6]}, + * {"ndistinct": 11, "attributes": [3,4,6]}] + * + * and + * {"3 =3D> 4": 1.000000, "3 =3D> 6": 1.000000, "4 =3D> 6": 1.000000, + * "3, 4 =3D> 6": 1.000000, "3, 6 =3D> 4": 1.000000} + * + * to + * + * [{"degree": 1.000000, "attributes": [3], "dependency": 4}, + * {"degree": 1.000000, "attributes": [3], "dependency": 6}, + * {"degree": 1.000000, "attributes": [4], "dependency": 6}, + * {"degree": 1.000000, "attributes": [3,4], "dependency": 6}, + * {"degree": 1.000000, "attributes": [3,6], "dependency": 4}] + */ + if (fout->remoteVersion >=3D 190000) + appendPQExpBufferStr(pq, "e.n_distinct, e.dependencies, "); + else + appendPQExpBufferStr(pq, + "( " + "SELECT json_agg( " + " json_build_object( " + " 'attributes', " + " string_to_array(kv.key, ', ')::integer[], " + " 'ndistinct', " + " kv.value::bigint )) " + "FROM json_each_text(e.n_distinct::text::json) AS kv" + ") AS n_distinct, " + "( " + "SELECT json_agg( " + " json_build_object( " + " 'attributes', " + " string_to_array( " + " split_part(kv.key, ' =3D> ', 1), " + " ', ')::integer[], " + " 'dependency', " + " split_part(kv.key, ' =3D> ', 2)::integer, " + " 'degree', " + " kv.value::double precision )) " + "FROM json_each_text(e.dependencies::text::json) AS kv " + ") AS dependencies, "); + + /* Versions < 12 do not have MCV */ + if (fout->remoteVersion >=3D 130000) + appendPQExpBufferStr(pq, + "e.most_common_vals, e.most_common_val_nulls, " + "e.most_common_freqs, e.most_common_base_freqs, "); + else + appendPQExpBufferStr(pq, + "NULL AS most_common_vals, NULL AS most_common_val_nulls, " + "NULL AS most_common_freqs, NULL AS most_common_base_freqs, "); + + /* Expressions were introduced in v14 */ + if (fout->remoteVersion >=3D 140000) + { + appendPQExpBufferStr(pq, + "( " + "SELECT array_agg( " + " ARRAY[ee.null_frac::text, ee.avg_width::text, " + " ee.n_distinct::text, ee.most_common_vals::text, " + " ee.most_common_freqs::text, ee.histogram_bounds::text,= " + " ee.correlation::text, ee.most_common_elems::text, " + " ee.most_common_elem_freqs::text, " + " ee.elem_count_histogram::text]) " + "FROM pg_stats_ext_exprs AS ee " + "WHERE ee.statistics_schemaname =3D $1 " + "AND ee.statistics_name =3D $2 "); + + /* Inherited expressions introduced in v15 */ + if (fout->remoteVersion >=3D 150000) + appendPQExpBufferStr(pq, "AND ee.inherited =3D e.inherited"); + + appendPQExpBufferStr(pq, ") AS exprs "); + } + else + appendPQExpBufferStr(pq, "NULL AS exprs "); + + /* pg_stats_ext introduced in v12 */ + if (fout->remoteVersion >=3D 120000) + appendPQExpBufferStr(pq, + "FROM pg_catalog.pg_stats_ext AS e " + "WHERE e.statistics_schemaname =3D $1 " + "AND e.statistics_name =3D $2 "); + else + appendPQExpBufferStr(pq, + "FROM ( " + "SELECT s.stxndistinct AS n_distinct, " + " s.stxdependencies AS dependencies " + "FROM pg_catalog.pg_statistics_ext AS s " + "JOIN pg_catalog.pg_namespace AS n " + "ON n.oid =3D s.stxnamespace " + "WHERE n.nspname =3D $1 " + "AND e.stxname =3D $2 " + ") AS e "); + + /* we always have an inherited column, but it may be a constant */ + appendPQExpBufferStr(pq, "ORDER BY inherited"); + + ExecuteSqlStatement(fout, pq->data); + + fout->is_prepared[PREPQUERY_DUMPEXTSTATSSTATS] =3D true; + + destroyPQExpBuffer(pq); + } + + query =3D createPQExpBuffer(); + + appendPQExpBufferStr(query, "EXECUTE getExtStatsStats("); + appendStringLiteralAH(query, statsextinfo->dobj.namespace->dobj.name, fou= t); + appendPQExpBufferStr(query, "::pg_catalog.name, "); + appendStringLiteralAH(query, statsextinfo->dobj.name, fout); + appendPQExpBufferStr(query, "::pg_catalog.name)"); + + res =3D ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + destroyPQExpBuffer(query); + + nstats =3D PQntuples(res); + + if (nstats > 0) + { + PQExpBuffer out =3D createPQExpBuffer(); + + int i_inherited =3D PQfnumber(res, "inherited"); + int i_ndistinct =3D PQfnumber(res, "n_distinct"); + int i_dependencies =3D PQfnumber(res, "dependencies"); + int i_mcv =3D PQfnumber(res, "most_common_vals"); + int i_mcv_nulls =3D PQfnumber(res, "most_common_val_nulls"); + int i_mcf =3D PQfnumber(res, "most_common_freqs"); + int i_mcbf =3D PQfnumber(res, "most_common_base_freqs"); + int i_exprs =3D PQfnumber(res, "exprs"); + + for (int i =3D 0; i < nstats; i++) + { + if (PQgetisnull(res, i, i_inherited)) + pg_fatal("inherited cannot be NULL"); + + appendPQExpBufferStr(out, + "SELECT * FROM pg_catalog.pg_restore_extended_stats(\n"); + appendPQExpBuffer(out, "\t'version', '%d'::integer,\n", + fout->remoteVersion); + appendPQExpBufferStr(out, "\t'statistics_schemaname', "); + appendStringLiteralAH(out, statsextinfo->dobj.namespace->dobj.name, fou= t); + appendPQExpBufferStr(out, ",\n\t'statistics_name', "); + appendStringLiteralAH(out, statsextinfo->dobj.name, fout); + appendNamedArgument(out, fout, "inherited", "boolean", + PQgetvalue(res, i, i_inherited)); + + if (!PQgetisnull(res, i, i_ndistinct)) + appendNamedArgument(out, fout, "n_distinct", "pg_ndistinct", + PQgetvalue(res, i, i_ndistinct)); + + if (!PQgetisnull(res, i, i_dependencies)) + appendNamedArgument(out, fout, "dependencies", "pg_dependencies", + PQgetvalue(res, i, i_dependencies)); + + if (!PQgetisnull(res, i, i_mcv)) + appendNamedArgument(out, fout, "most_common_vals", "text[]", + PQgetvalue(res, i, i_mcv)); + + if (!PQgetisnull(res, i, i_mcv_nulls)) + appendNamedArgument(out, fout, "most_common_val_nulls", "boolean[]", + PQgetvalue(res, i, i_mcv_nulls)); + + if (!PQgetisnull(res, i, i_mcf)) + appendNamedArgument(out, fout, "most_common_freqs", "double precision[= ]", + PQgetvalue(res, i, i_mcf)); + + if (!PQgetisnull(res, i, i_mcbf)) + appendNamedArgument(out, fout, "most_common_base_freqs", "double preci= sion[]", + PQgetvalue(res, i, i_mcbf)); + + if (!PQgetisnull(res, i, i_exprs)) + appendNamedArgument(out, fout, "exprs", "text[]", + PQgetvalue(res, i, i_exprs)); + + appendPQExpBufferStr(out, "\n);\n"); + } + + ArchiveEntry(fout, nilCatalogId, createDumpId(), + ARCHIVE_OPTS(.tag =3D statsextinfo->dobj.name, + .namespace =3D statsextinfo->dobj.namespace->dobj.name, + .owner =3D statsextinfo->rolname, + .description =3D "EXTENDED STATISTICS DATA", + .section =3D SECTION_POST_DATA, + .createStmt =3D out->data, + .deps =3D &statsextinfo->dobj.dumpId, + .nDeps =3D 1)); + destroyPQExpBuffer(out); + } + PQclear(res); +} + /* * dumpConstraint * write out to fout a user-defined constraint diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_du= mp.pl index 445a541abf63..6681265974f6 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -4772,6 +4772,34 @@ my %tests =3D ( }, }, =20 + # + # EXTENDED stats will end up in SECTION_POST_DATA. + # + 'extended_statistics_import' =3D> { + create_sql =3D> ' + CREATE TABLE dump_test.has_ext_stats + AS SELECT g.g AS x, g.g / 2 AS y FROM generate_series(1,100) AS g(g); + CREATE STATISTICS dump_test.es1 ON x, (y % 2) FROM dump_test.has_ext_st= ats; + ANALYZE dump_test.has_ext_stats;', + regexp =3D> qr/^ + \QSELECT * FROM pg_catalog.pg_restore_extended_stats(\E\s+/xm, + like =3D> { + %full_runs, + %dump_test_schema_runs, + no_data_no_schema =3D> 1, + no_schema =3D> 1, + section_post_data =3D> 1, + statistics_only =3D> 1, + schema_only_with_statistics =3D> 1, + }, + unlike =3D> { + exclude_dump_test_schema =3D> 1, + no_statistics =3D> 1, + only_dump_measurement =3D> 1, + schema_only =3D> 1, + }, + }, + # # While attribute stats (aka pg_statistic stats) only appear for tables # that have been analyzed, all tables will have relation stats because --=20 2.51.0 --AorqrXjCfzfo1Gz1-- --eH7/90dwF2tDSx+7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmkaxwYACgkQnvQgOdby QH22TRAAoRtPJvU8IQ7a1/IfibvdKenisvcmn7Dv8XfwbcOuc/Leap3JpB2k7xUv xoB9+tDI6+WXt4Nqg+63Et2Jhi6YV28S0gaducVDCT9lHifwd+4vDdRNnAC5c90x GJHgfE0vMXmEE3oJ4SjpRqOzH+tC3bjx815U1oxkUgHmgs1CFX5lISgAImHIqU8v DeWuvbNk0dQfuryrKaFxT6+e+DklK15a63uWvGqCnF+rYgCMEfZkKOsdTR1hM3So SjprmXwTfWUMdiAjeVHVzgWli+3h+VM7oPcNrs3DRu9zcrvOzzgzhDKPcoOLpYN9 LnFS+dNkyH9PT1CgSOIjPjZVorw4vtjnB9jud0iYLgxHmp2FA43zzkdzq75Is0Ro H0D4yP+LAaM6aR6fQDpAnylV6dDLZ6ka4yp78uG1yOvGi7uu9tZvWe1RhrzaUfkl 6J/pW6pXDeSTzZtGwIhiCZ2oj13UHi1IhbkmhaWS2ErA8eKd1ie8S1VLwJpme9y8 bKiP+vFDi3KMMsxKNRY0q8ZjVR3Y62yGFT1Mdun3d9SBbwgRmuxFdNwX5zt+LMTs 8260YddM/qev+3TPTO8pF8/526KaiqOZGsrRQHIkPX9NtkCKqzC0zypeHujepPzn 3CFZlebBcxs4i1CLswLHaZP23DdJ2wN3gPC99SWjXqXPpr4+zDI= =T7Pg -----END PGP SIGNATURE----- --eH7/90dwF2tDSx+7--