Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1j9XRY-0003PO-IY for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Mar 2020 17:04:25 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1j9XRW-00021R-9n for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Mar 2020 17:04:22 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from <9erthalion6@gmail.com>) id 1j9XQN-0001jp-Lz for pgsql-hackers@lists.postgresql.org; Wed, 04 Mar 2020 17:04:21 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <9erthalion6@gmail.com>) id 1j9XQ1-0004yw-6S for pgsql-hackers@postgresql.org; Wed, 04 Mar 2020 17:03:00 +0000 Received: by mail-wm1-x342.google.com with SMTP id e26so2626930wme.5 for ; Wed, 04 Mar 2020 09:02:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=7Fnk1OEFUSZcVmjLzazUraUf82plXjKt4rc1Rakq1yg=; b=PNRXxHLfk5xWEz3q1t44EZykLp2UMEKi+WehpSEfYIRAYuBXNDEdgAekAbrqA03hQY D+E0lA1yzxvt8O+Xubvpn7Uowu95RCrzUD4JSTNFjhnQOWocaf0ZWxtnFQsd2gFlYxyu MuIYcxx/ejKkWTuvgbFEJ0E5rhn/1lytpRELyn3Bp0516kf/dA6ACvk52W8iOyMZ6qI0 t/HEmM5cOnAzlOlQsLSFXumavtoFISqw11b9tzP5d5Qwo3wvbpVGlJ82wsaXSqv2trZQ 1vcCvi7hTWGbL94foCI2558osF1c+6RQNWxLai2kXrpvsFvW9RzNYOeuX/ROuSHs5X7W HZlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=7Fnk1OEFUSZcVmjLzazUraUf82plXjKt4rc1Rakq1yg=; b=kuc/AUQWbAPUMhed4xShaCL/9nv1P9yTNL/GUuiueS/XhVr6bJcPq2jluy2mCYu08D 4URKhcp8sT3nWgm1rzAgBSzBoVSpbAPzBvzDFX2TMGEcAsUDL7hrncqYQMMQ0uXXGaAQ YmSNdjDlRkBNnwy23YumIeZOnXE/jgo7GhDgynmWfC5qF23xCoeaDXMFq5bzvet2SqFt MaUIKIVJ8/qPawS3wfB6IUs2XlD7RE765fc9AOjd8lHd5M7TQ+LemAWZ1DN17yaYRAHS kUut/wLhW5mKvYiikZ4bl46yBXi8VVzLSj66Bpm5JqTPFGpqD4bwDzHItQbVgxRHb9+N yCTA== X-Gm-Message-State: ANhLgQ1DZqOFM48pTWhHM5RHpzRb2MEMRXcpAmTfOmnY+26qXn/Qxv/N a6R120LJtiQQ84iB003Rimk= X-Google-Smtp-Source: ADFU+vsnmL4NsIUNtpLeVTfJyWo61ewFj3yvili6MHQaIheHSbZ5DJYGqfilvYegE/DGZYkXm+/Muw== X-Received: by 2002:a1c:c2c5:: with SMTP id s188mr4408218wmf.162.1583341367665; Wed, 04 Mar 2020 09:02:47 -0800 (PST) Received: from localhost (dslb-094-222-004-201.094.222.pools.vodafone-ip.de. [94.222.4.201]) by smtp.gmail.com with ESMTPSA id m25sm4832236wml.35.2020.03.04.09.02.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2020 09:02:46 -0800 (PST) Date: Wed, 4 Mar 2020 18:03:39 +0100 From: Dmitry Dolgov <9erthalion6@gmail.com> To: David Steele Cc: Pavel Stehule , Nikita Glukhov , Alvaro Herrera , David Fetter , Thomas Munro , Thomas Munro , Oleksandr Shulgin , Tom Lane , Robert Haas , Andres Freund , Michael Paquier , Oleg Bartunov , Peter Eisentraut , PostgreSQL-development Subject: Re: [HACKERS] [PATCH] Generic type subscripting Message-ID: <20200304170339.g4xgydtxtmgfajbe@localhost> References: <20190925202205.GA23893@alvherre.pgsql> <099309f0-4b41-2c78-aac8-0835fe9a9ee7@postgrespro.ru> <20191110123208.r4kn5k34wsey5zik@localhost> <20191219142030.uc6en2ci3rx2jc4c@localhost> <20200213131228.i6afwfmjdod7obko@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="g5st7yssq7xavhfb" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --g5st7yssq7xavhfb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline > On Tue, Mar 03, 2020 at 12:55:38PM -0500, David Steele wrote: > > > Yep, I wasn't paying much attention recently to this patch, will post > > rebased and fixed version soon. > > The last CF for PG13 has begun. Do you know when you'll have a rebased and > updated patch available? The current patch no longer applies. Thanks for the reminder! Here is the rebased version, with one extra detail. Looks like tests for non-strict version of jsonb_set are doing: \pset null NULL and backwards, but via: \pset null which is not changing it back. It's showed up in tests for subscripting down the file, so I've changed it to what I guess it was suppose to be: \pset null '' --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0001-Base-implementation-of-subscripting-mechanism.patch" From 5bab75957e8cfd5d69f35a2971a00573f8d6f7a8 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Thu, 31 Jan 2019 22:37:19 +0100 Subject: [PATCH v30 1/6] Base implementation of subscripting mechanism Introduce all the required machinery for generalizing subscripting operation for a different data types: * subscripting handler procedure, to set up a relation between data type and corresponding subscripting logic. * subscripting routines, that help generalize a subscripting logic, since it involves few stages, namely preparation (e.g. to figure out required types), validation (to check the input and return meaningful error message), fetch (executed when we extract a value using subscripting), assign (executed when we update a data type with a new value using subscripting). Without this it would be neccessary to introduce more new fields to pg_type, which would be too invasive. All ArrayRef related logic was removed and landed as a separate subscripting implementation in the following patch from this series. The rest of the code was rearranged, to e.g. store a type of assigned value for an assign operation. Reviewed-by: Tom Lane, Arthur Zakirov --- .../pg_stat_statements/pg_stat_statements.c | 1 + src/backend/catalog/heap.c | 6 +- src/backend/catalog/pg_type.c | 15 +- src/backend/commands/typecmds.c | 77 +++++++++- src/backend/executor/execExpr.c | 25 +--- src/backend/executor/execExprInterp.c | 124 +++------------ src/backend/nodes/copyfuncs.c | 2 + src/backend/nodes/equalfuncs.c | 2 + src/backend/nodes/outfuncs.c | 2 + src/backend/nodes/readfuncs.c | 2 + src/backend/parser/parse_expr.c | 54 ++++--- src/backend/parser/parse_node.c | 141 ++++-------------- src/backend/parser/parse_target.c | 88 +++++------ src/backend/utils/adt/ruleutils.c | 21 +-- src/backend/utils/cache/lsyscache.c | 23 +++ src/include/c.h | 2 + src/include/catalog/pg_type.h | 9 +- src/include/executor/execExpr.h | 13 +- src/include/nodes/primnodes.h | 6 + src/include/nodes/subscripting.h | 42 ++++++ src/include/parser/parse_node.h | 4 +- src/include/utils/lsyscache.h | 1 + 22 files changed, 334 insertions(+), 326 deletions(-) create mode 100644 src/include/nodes/subscripting.h diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 20dc8c605b..728f781620 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c @@ -2597,6 +2597,7 @@ JumbleExpr(pgssJumbleState *jstate, Node *node) JumbleExpr(jstate, (Node *) sbsref->reflowerindexpr); JumbleExpr(jstate, (Node *) sbsref->refexpr); JumbleExpr(jstate, (Node *) sbsref->refassgnexpr); + APP_JUMB(sbsref->refnestedfunc); } break; case T_FuncExpr: diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index 9d9e915979..db746c99b6 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -1058,7 +1058,8 @@ AddNewRelationType(const char *typeName, -1, /* typmod */ 0, /* array dimensions for typBaseType */ false, /* Type NOT NULL */ - InvalidOid); /* rowtypes never have a collation */ + InvalidOid, /* rowtypes never have a collation */ + InvalidOid); /* typsubshandler - none */ } /* -------------------------------- @@ -1339,7 +1340,8 @@ heap_create_with_catalog(const char *relname, -1, /* typmod */ 0, /* array dimensions for typBaseType */ false, /* Type NOT NULL */ - InvalidOid); /* rowtypes never have a collation */ + InvalidOid, /* rowtypes never have a collation */ + 0); /* array implementation */ pfree(relarrayname); } diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c index 56e0bcf39e..22e8a46695 100644 --- a/src/backend/catalog/pg_type.c +++ b/src/backend/catalog/pg_type.c @@ -118,6 +118,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId) values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(-1); values[Anum_pg_type_typndims - 1] = Int32GetDatum(0); values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(InvalidOid); + values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(InvalidOid); nulls[Anum_pg_type_typdefaultbin - 1] = true; nulls[Anum_pg_type_typdefault - 1] = true; nulls[Anum_pg_type_typacl - 1] = true; @@ -158,10 +159,10 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId) GenerateTypeDependencies(typoid, (Form_pg_type) GETSTRUCT(tup), NULL, - NULL, 0, false, false, + InvalidOid, false); /* Post creation hook for new shell type */ @@ -219,7 +220,8 @@ TypeCreate(Oid newTypeOid, int32 typeMod, int32 typNDims, /* Array dimensions for baseType */ bool typeNotNull, - Oid typeCollation) + Oid typeCollation, + Oid subscriptingHandlerProcedure) { Relation pg_type_desc; Oid typeObjectId; @@ -372,6 +374,7 @@ TypeCreate(Oid newTypeOid, values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(typeMod); values[Anum_pg_type_typndims - 1] = Int32GetDatum(typNDims); values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(typeCollation); + values[Anum_pg_type_typsubshandler - 1] = ObjectIdGetDatum(subscriptingHandlerProcedure); /* * initialize the default binary value for this type. Check for nulls of @@ -695,6 +698,14 @@ GenerateTypeDependencies(Oid typeObjectId, /* Normal dependency on the default expression. */ if (defaultExpr) recordDependencyOnExpr(&myself, defaultExpr, NIL, DEPENDENCY_NORMAL); + + if (OidIsValid(typeForm->typsubshandler)) + { + referenced.classId = ProcedureRelationId; + referenced.objectId = typeForm->typsubshandler; + referenced.objectSubId = 0; + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + } } /* diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c index 99528bf1d5..660e6df39a 100644 --- a/src/backend/commands/typecmds.c +++ b/src/backend/commands/typecmds.c @@ -95,6 +95,7 @@ static Oid findTypeSendFunction(List *procname, Oid typeOid); static Oid findTypeTypmodinFunction(List *procname); static Oid findTypeTypmodoutFunction(List *procname); static Oid findTypeAnalyzeFunction(List *procname, Oid typeOid); +static Oid findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc); static Oid findRangeSubOpclass(List *opcname, Oid subtype); static Oid findRangeCanonicalFunction(List *procname, Oid typeOid); static Oid findRangeSubtypeDiffFunction(List *procname, Oid subtype); @@ -126,6 +127,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) List *typmodinName = NIL; List *typmodoutName = NIL; List *analyzeName = NIL; + List *subscriptingParseName = NIL; char category = TYPCATEGORY_USER; bool preferred = false; char delimiter = DEFAULT_TYPDELIM; @@ -144,6 +146,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) DefElem *typmodinNameEl = NULL; DefElem *typmodoutNameEl = NULL; DefElem *analyzeNameEl = NULL; + DefElem *subscriptingParseNameEl = NULL; DefElem *categoryEl = NULL; DefElem *preferredEl = NULL; DefElem *delimiterEl = NULL; @@ -166,6 +169,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) Oid resulttype; ListCell *pl; ObjectAddress address; + Oid subscriptingParseOid = InvalidOid; /* * As of Postgres 8.4, we require superuser privilege to create a base @@ -265,6 +269,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) else if (strcmp(defel->defname, "analyze") == 0 || strcmp(defel->defname, "analyse") == 0) defelp = &analyzeNameEl; + else if (strcmp(defel->defname, "subscripting_handler") == 0) + defelp = &subscriptingParseNameEl; else if (strcmp(defel->defname, "category") == 0) defelp = &categoryEl; else if (strcmp(defel->defname, "preferred") == 0) @@ -335,6 +341,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) typmodoutName = defGetQualifiedName(typmodoutNameEl); if (analyzeNameEl) analyzeName = defGetQualifiedName(analyzeNameEl); + if (subscriptingParseNameEl) + subscriptingParseName = defGetQualifiedName(subscriptingParseNameEl); if (categoryEl) { char *p = defGetString(categoryEl); @@ -516,6 +524,10 @@ DefineType(ParseState *pstate, List *names, List *parameters) if (analyzeName) analyzeOid = findTypeAnalyzeFunction(analyzeName, typoid); + if (subscriptingParseName) + subscriptingParseOid = findTypeSubscriptingFunction(subscriptingParseName, + typoid, true); + /* * Check permissions on functions. We choose to require the creator/owner * of a type to also own the underlying functions. Since creating a type @@ -635,7 +647,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) -1, /* typMod (Domains only) */ 0, /* Array Dimensions of typbasetype */ false, /* Type NOT NULL */ - collation); /* type's collation */ + collation, /* type's collation */ + subscriptingParseOid); /* subscripting procedure */ Assert(typoid == address.objectId); /* @@ -676,7 +689,8 @@ DefineType(ParseState *pstate, List *names, List *parameters) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - collation); /* type's collation */ + collation, /* type's collation */ + 0); pfree(array_type); @@ -739,6 +753,7 @@ DefineDomain(CreateDomainStmt *stmt) Oid receiveProcedure; Oid sendProcedure; Oid analyzeProcedure; + Oid subscriptingHandlerProcedure; bool byValue; char category; char delimiter; @@ -866,6 +881,9 @@ DefineDomain(CreateDomainStmt *stmt) /* Analysis function */ analyzeProcedure = baseType->typanalyze; + /* Subscripting functions */ + subscriptingHandlerProcedure = baseType->typsubshandler; + /* Inherited default value */ datum = SysCacheGetAttr(TYPEOID, typeTup, Anum_pg_type_typdefault, &isnull); @@ -1071,7 +1089,8 @@ DefineDomain(CreateDomainStmt *stmt) basetypeMod, /* typeMod value */ typNDims, /* Array dimensions for base type */ typNotNull, /* Type NOT NULL */ - domaincoll); /* type's collation */ + domaincoll, /* type's collation */ + subscriptingHandlerProcedure); /* subscripting procedure */ /* * Create the array type that goes with it. @@ -1111,7 +1130,8 @@ DefineDomain(CreateDomainStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - domaincoll); /* type's collation */ + domaincoll, /* type's collation */ + 0); /* array subscripting implementation */ pfree(domainArrayName); @@ -1226,7 +1246,8 @@ DefineEnum(CreateEnumStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation */ + InvalidOid, /* type's collation */ + InvalidOid); /* typsubshandler - none */ /* Enter the enum's values into pg_enum */ EnumValuesCreate(enumTypeAddr.objectId, stmt->vals); @@ -1266,7 +1287,8 @@ DefineEnum(CreateEnumStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation */ + InvalidOid, /* type's collation */ + 0); /* array subscripting implementation */ pfree(enumArrayName); @@ -1554,7 +1576,8 @@ DefineRange(CreateRangeStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* type's collation (ranges never have one) */ + InvalidOid, /* type's collation (ranges never have one) */ + InvalidOid); /* typsubshandler - none */ Assert(typoid == address.objectId); /* Create the entry in pg_range */ @@ -1596,7 +1619,8 @@ DefineRange(CreateRangeStmt *stmt) -1, /* typMod (Domains only) */ 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ - InvalidOid); /* typcollation */ + InvalidOid, /* typcollation */ + 0); /* array subscripting implementation */ pfree(rangeArrayName); @@ -1940,6 +1964,43 @@ findTypeAnalyzeFunction(List *procname, Oid typeOid) return procOid; } +static Oid +findTypeSubscriptingFunction(List *procname, Oid typeOid, bool parseFunc) +{ + Oid argList[2]; + Oid procOid; + int nargs; + + if (parseFunc) + { + /* + * Subscripting function parse always take two INTERNAL argument and + * return INTERNAL. + */ + argList[0] = INTERNALOID; + nargs = 1; + } + else + { + /* + * Subscripting functions fetch/assign always take one typeOid + * argument, one INTERNAL argument and return typeOid. + */ + argList[0] = typeOid; + argList[1] = INTERNALOID; + nargs = 2; + } + + procOid = LookupFuncName(procname, nargs, argList, true); + if (!OidIsValid(procOid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("function %s does not exist", + func_signature_string(procname, nargs, NIL, argList)))); + + return procOid; +} + /* * Find suitable support functions and opclasses for a range type. */ diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c index 91aa386fa6..a756f4ba8d 100644 --- a/src/backend/executor/execExpr.c +++ b/src/backend/executor/execExpr.c @@ -2543,18 +2543,16 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, { bool isAssignment = (sbsref->refassgnexpr != NULL); SubscriptingRefState *sbsrefstate = palloc0(sizeof(SubscriptingRefState)); - List *adjust_jumps = NIL; - ListCell *lc; - int i; + List *adjust_jumps = NIL; + ListCell *lc; + int i; + RegProcedure typsubshandler = get_typsubsprocs(sbsref->refcontainertype); /* Fill constant fields of SubscriptingRefState */ sbsrefstate->isassignment = isAssignment; sbsrefstate->refelemtype = sbsref->refelemtype; sbsrefstate->refattrlength = get_typlen(sbsref->refcontainertype); - get_typlenbyvalalign(sbsref->refelemtype, - &sbsrefstate->refelemlength, - &sbsrefstate->refelembyval, - &sbsrefstate->refelemalign); + sbsrefstate->sbsroutines = (SubscriptRoutines *) OidFunctionCall0(typsubshandler); /* * Evaluate array input. It's safe to do so into resv/resnull, because we @@ -2578,19 +2576,6 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, state->steps_len - 1); } - /* Verify subscript list lengths are within limit */ - if (list_length(sbsref->refupperindexpr) > MAXDIM) - ereport(ERROR, - (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), - errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", - list_length(sbsref->refupperindexpr), MAXDIM))); - - if (list_length(sbsref->reflowerindexpr) > MAXDIM) - ereport(ERROR, - (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), - errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", - list_length(sbsref->reflowerindexpr), MAXDIM))); - /* Evaluate upper subscripts */ i = 0; foreach(lc, sbsref->refupperindexpr) diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c index eafd484900..a945cad9d3 100644 --- a/src/backend/executor/execExprInterp.c +++ b/src/backend/executor/execExprInterp.c @@ -3134,8 +3134,8 @@ bool ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; - int *indexes; - int off; + Datum *indexes; + int off; /* If any index expr yields NULL, result is NULL or error */ if (sbsrefstate->subscriptnull) @@ -3155,7 +3155,7 @@ ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) indexes = sbsrefstate->lowerindex; off = op->d.sbsref_subscript.off; - indexes[off] = DatumGetInt32(sbsrefstate->subscriptvalue); + indexes[off] = sbsrefstate->subscriptvalue; return true; } @@ -3169,36 +3169,14 @@ void ExecEvalSubscriptingRefFetch(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; /* Should not get here if source container (or any subscript) is null */ Assert(!(*op->resnull)); - if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - *op->resvalue = array_get_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign, - op->resnull); - } - else - { - /* Slice case */ - *op->resvalue = array_get_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } + + *op->resvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate); + *op->resnull = sbsrefstate->resnull; } /* @@ -3211,40 +3189,20 @@ void ExecEvalSubscriptingRefOld(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; if (*op->resnull) { - /* whole array is null, so any element or slice is too */ + /* whole container is null, so any element or slice is too */ sbsrefstate->prevvalue = (Datum) 0; sbsrefstate->prevnull = true; } - else if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - sbsrefstate->prevvalue = array_get_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign, - &sbsrefstate->prevnull); - } else { - /* Slice case */ - /* this is currently unreachable */ - sbsrefstate->prevvalue = array_get_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - sbsrefstate->prevnull = false; + sbsrefstate->prevvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate); + + if (sbsrefstate->numlower != 0) + sbsrefstate->prevnull = false; } } @@ -3258,59 +3216,11 @@ void ExecEvalSubscriptingRefAssign(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; - /* - * For an assignment to a fixed-length container type, both the original - * container and the value to be assigned into it must be non-NULL, else - * we punt and return the original container. - */ - if (sbsrefstate->refattrlength > 0) - { - if (*op->resnull || sbsrefstate->replacenull) - return; - } - - /* - * For assignment to varlena arrays, we handle a NULL original array by - * substituting an empty (zero-dimensional) array; insertion of the new - * element will result in a singleton array value. It does not matter - * whether the new element is NULL. - */ - if (*op->resnull) - { - *op->resvalue = PointerGetDatum(construct_empty_array(sbsrefstate->refelemtype)); - *op->resnull = false; - } - - if (sbsrefstate->numlower == 0) - { - /* Scalar case */ - *op->resvalue = array_set_element(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->replacevalue, - sbsrefstate->replacenull, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } - else - { - /* Slice case */ - *op->resvalue = array_set_slice(*op->resvalue, - sbsrefstate->numupper, - sbsrefstate->upperindex, - sbsrefstate->lowerindex, - sbsrefstate->upperprovided, - sbsrefstate->lowerprovided, - sbsrefstate->replacevalue, - sbsrefstate->replacenull, - sbsrefstate->refattrlength, - sbsrefstate->refelemlength, - sbsrefstate->refelembyval, - sbsrefstate->refelemalign); - } + sbsrefstate->resnull = *op->resnull; + *op->resvalue = sbsroutines->assign(*op->resvalue, sbsrefstate); + *op->resnull = sbsrefstate->resnull; } /* diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index e04c33e4ad..d3a15dc0a5 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -1509,8 +1509,10 @@ _copySubscriptingRef(const SubscriptingRef *from) COPY_SCALAR_FIELD(refcontainertype); COPY_SCALAR_FIELD(refelemtype); + COPY_SCALAR_FIELD(refassgntype); COPY_SCALAR_FIELD(reftypmod); COPY_SCALAR_FIELD(refcollid); + COPY_SCALAR_FIELD(refnestedfunc); COPY_NODE_FIELD(refupperindexpr); COPY_NODE_FIELD(reflowerindexpr); COPY_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index 5b1ba143b1..e1d23e54d5 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -274,8 +274,10 @@ _equalSubscriptingRef(const SubscriptingRef *a, const SubscriptingRef *b) { COMPARE_SCALAR_FIELD(refcontainertype); COMPARE_SCALAR_FIELD(refelemtype); + COMPARE_SCALAR_FIELD(refassgntype); COMPARE_SCALAR_FIELD(reftypmod); COMPARE_SCALAR_FIELD(refcollid); + COMPARE_SCALAR_FIELD(refnestedfunc); COMPARE_NODE_FIELD(refupperindexpr); COMPARE_NODE_FIELD(reflowerindexpr); COMPARE_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index e084c3f069..5bc53474c6 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -1173,8 +1173,10 @@ _outSubscriptingRef(StringInfo str, const SubscriptingRef *node) WRITE_OID_FIELD(refcontainertype); WRITE_OID_FIELD(refelemtype); + WRITE_OID_FIELD(refassgntype); WRITE_INT_FIELD(reftypmod); WRITE_OID_FIELD(refcollid); + WRITE_OID_FIELD(refnestedfunc); WRITE_NODE_FIELD(refupperindexpr); WRITE_NODE_FIELD(reflowerindexpr); WRITE_NODE_FIELD(refexpr); diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index d5b23a3479..f537a30772 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -668,8 +668,10 @@ _readSubscriptingRef(void) READ_OID_FIELD(refcontainertype); READ_OID_FIELD(refelemtype); + READ_OID_FIELD(refassgntype); READ_INT_FIELD(reftypmod); READ_OID_FIELD(refcollid); + READ_OID_FIELD(refnestedfunc); READ_NODE_FIELD(refupperindexpr); READ_NODE_FIELD(reflowerindexpr); READ_NODE_FIELD(refexpr); diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 831db4af95..3173277e44 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -433,11 +433,13 @@ unknown_attribute(ParseState *pstate, Node *relref, const char *attname, static Node * transformIndirection(ParseState *pstate, A_Indirection *ind) { - Node *last_srf = pstate->p_last_srf; - Node *result = transformExprRecurse(pstate, ind->arg); - List *subscripts = NIL; - int location = exprLocation(result); - ListCell *i; + Node *last_srf = pstate->p_last_srf; + Node *result = transformExprRecurse(pstate, ind->arg); + SubscriptRoutines *sbsroutines; + SubscriptingRef *sbsref; + List *subscripts = NIL; + int location = exprLocation(result); + ListCell *i; /* * We have to split any field-selection operations apart from @@ -465,13 +467,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind) /* process subscripts before this field selection */ if (subscripts) - result = (Node *) transformContainerSubscripts(pstate, - result, - exprType(result), - InvalidOid, - exprTypmod(result), - subscripts, - NULL); + { + sbsref = transformContainerSubscripts(pstate, + result, + exprType(result), + InvalidOid, + exprTypmod(result), + subscripts, + NULL); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + sbsref = sbsroutines->prepare(false, sbsref); + sbsroutines->validate(false, sbsref, pstate); + result = (Node *) sbsref; + } subscripts = NIL; newresult = ParseFuncOrColumn(pstate, @@ -488,13 +497,20 @@ transformIndirection(ParseState *pstate, A_Indirection *ind) } /* process trailing subscripts, if any */ if (subscripts) - result = (Node *) transformContainerSubscripts(pstate, - result, - exprType(result), - InvalidOid, - exprTypmod(result), - subscripts, - NULL); + { + sbsref = transformContainerSubscripts(pstate, + result, + exprType(result), + InvalidOid, + exprTypmod(result), + subscripts, + NULL); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + sbsref = sbsroutines->prepare(false, sbsref); + sbsroutines->validate(false, sbsref, pstate); + result = (Node *) sbsref; + } return result; } diff --git a/src/backend/parser/parse_node.c b/src/backend/parser/parse_node.c index 6e98fe55fc..12dde390d3 100644 --- a/src/backend/parser/parse_node.c +++ b/src/backend/parser/parse_node.c @@ -184,21 +184,12 @@ pcb_error_callback(void *arg) * transformContainerType() * Identify the types involved in a subscripting operation for container * - * - * On entry, containerType/containerTypmod identify the type of the input value - * to be subscripted (which could be a domain type). These are modified if - * necessary to identify the actual container type and typmod, and the - * container's element type is returned. An error is thrown if the input isn't - * an array type. + * On entry, containerType/containerTypmod are modified if necessary to + * identify the actual container type and typmod. */ -Oid +void transformContainerType(Oid *containerType, int32 *containerTypmod) { - Oid origContainerType = *containerType; - Oid elementType; - HeapTuple type_tuple_container; - Form_pg_type type_struct_container; - /* * If the input is a domain, smash to base type, and extract the actual * typmod to be applied to the base type. Subscripting a domain is an @@ -219,25 +210,6 @@ transformContainerType(Oid *containerType, int32 *containerTypmod) *containerType = INT2ARRAYOID; else if (*containerType == OIDVECTOROID) *containerType = OIDARRAYOID; - - /* Get the type tuple for the container */ - type_tuple_container = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*containerType)); - if (!HeapTupleIsValid(type_tuple_container)) - elog(ERROR, "cache lookup failed for type %u", *containerType); - type_struct_container = (Form_pg_type) GETSTRUCT(type_tuple_container); - - /* needn't check typisdefined since this will fail anyway */ - - elementType = type_struct_container->typelem; - if (elementType == InvalidOid) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("cannot subscript type %s because it is not an array", - format_type_be(origContainerType)))); - - ReleaseSysCache(type_tuple_container); - - return elementType; } /* @@ -254,10 +226,15 @@ transformContainerType(Oid *containerType, int32 *containerTypmod) * container. We produce an expression that represents the new container value * with the source data inserted into the right part of the container. * - * For both cases, if the source container is of a domain-over-array type, - * the result is of the base array type or its element type; essentially, - * we must fold a domain to its base type before applying subscripting. - * (Note that int2vector and oidvector are treated as domains here.) + * For both cases, this function contains only general subscripting logic while + * type-specific logic (e.g. type verifications and coersion) is placen in + * separate procedure indicated by typsubshandler. There is only one exception + * for now about domain-over-container, if the source container is of a + * domain-over-container type, the result is of the base container type or its + * element type; essentially, we must fold a domain to its base type before + * applying subscripting. (Note that int2vector and oidvector are treated as + * domains here.) An error will appear in case if current container type + * doesn't have a subscripting procedure. * * pstate Parse state * containerBase Already-transformed expression for the container as a whole @@ -284,16 +261,12 @@ transformContainerSubscripts(ParseState *pstate, bool isSlice = false; List *upperIndexpr = NIL; List *lowerIndexpr = NIL; + List *indexprSlice = NIL; ListCell *idx; SubscriptingRef *sbsref; - /* - * Caller may or may not have bothered to determine elementType. Note - * that if the caller did do so, containerType/containerTypMod must be as - * modified by transformContainerType, ie, smash domain to base type. - */ - if (!OidIsValid(elementType)) - elementType = transformContainerType(&containerType, &containerTypMod); + /* Identify the actual container type and element type involved */ + transformContainerType(&containerType, &containerTypMod); /* * A list containing only simple subscripts refers to a single container @@ -327,29 +300,6 @@ transformContainerSubscripts(ParseState *pstate, if (ai->lidx) { subexpr = transformExpr(pstate, ai->lidx, pstate->p_expr_kind); - /* If it's not int4 already, try to coerce */ - subexpr = coerce_to_target_type(pstate, - subexpr, exprType(subexpr), - INT4OID, -1, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (subexpr == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array subscript must have type integer"), - parser_errposition(pstate, exprLocation(ai->lidx)))); - } - else if (!ai->is_slice) - { - /* Make a constant 1 */ - subexpr = (Node *) makeConst(INT4OID, - -1, - InvalidOid, - sizeof(int32), - Int32GetDatum(1), - false, - true); /* pass by value */ } else { @@ -357,63 +307,23 @@ transformContainerSubscripts(ParseState *pstate, subexpr = NULL; } lowerIndexpr = lappend(lowerIndexpr, subexpr); + indexprSlice = lappend(indexprSlice, ai); } else Assert(ai->lidx == NULL && !ai->is_slice); if (ai->uidx) - { subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); - /* If it's not int4 already, try to coerce */ - subexpr = coerce_to_target_type(pstate, - subexpr, exprType(subexpr), - INT4OID, -1, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (subexpr == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array subscript must have type integer"), - parser_errposition(pstate, exprLocation(ai->uidx)))); - } else { /* Slice with omitted upper bound, put NULL into the list */ Assert(isSlice && ai->is_slice); subexpr = NULL; } + subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); upperIndexpr = lappend(upperIndexpr, subexpr); } - /* - * If doing an array store, coerce the source value to the right type. - * (This should agree with the coercion done by transformAssignedExpr.) - */ - if (assignFrom != NULL) - { - Oid typesource = exprType(assignFrom); - Oid typeneeded = isSlice ? containerType : elementType; - Node *newFrom; - - newFrom = coerce_to_target_type(pstate, - assignFrom, typesource, - typeneeded, containerTypMod, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (newFrom == NULL) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array assignment requires type %s" - " but expression is of type %s", - format_type_be(typeneeded), - format_type_be(typesource)), - errhint("You will need to rewrite or cast the expression."), - parser_errposition(pstate, exprLocation(assignFrom)))); - assignFrom = newFrom; - } - /* * Ready to build the SubscriptingRef node. */ @@ -422,17 +332,30 @@ transformContainerSubscripts(ParseState *pstate, sbsref->refassgnexpr = (Expr *) assignFrom; sbsref->refcontainertype = containerType; - sbsref->refelemtype = elementType; sbsref->reftypmod = containerTypMod; /* refcollid will be set by parse_collate.c */ sbsref->refupperindexpr = upperIndexpr; sbsref->reflowerindexpr = lowerIndexpr; + sbsref->refindexprslice = indexprSlice; sbsref->refexpr = (Expr *) containerBase; - sbsref->refassgnexpr = (Expr *) assignFrom; return sbsref; } +SubscriptRoutines* +getSubscriptingRoutines(Oid containerType) +{ + RegProcedure typsubshandler = get_typsubsprocs(containerType); + + if (!OidIsValid(typsubshandler)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot subscript type %s because it does not support subscripting", + format_type_be(containerType)))); + + return (SubscriptRoutines *) OidFunctionCall0(typsubshandler); +} + /* * make_const * diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c index 566c517837..a5e734837f 100644 --- a/src/backend/parser/parse_target.c +++ b/src/backend/parser/parse_target.c @@ -685,7 +685,7 @@ transformAssignmentIndirection(ParseState *pstate, Node *rhs, int location) { - Node *result; + Node *result = NULL; List *subscripts = NIL; bool isSlice = false; ListCell *i; @@ -848,27 +848,21 @@ transformAssignmentIndirection(ParseState *pstate, location); } - /* base case: just coerce RHS to match target type ID */ - - result = coerce_to_target_type(pstate, - rhs, exprType(rhs), - targetTypeId, targetTypMod, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (result == NULL) + /* + * Base case: just coerce RHS to match target type ID. + * It's necessary only for field selection, since for + * subscripting it's custom code who should define types. + */ + if (!targetIsSubscripting) { - if (targetIsSubscripting) - ereport(ERROR, - (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("array assignment to \"%s\" requires type %s" - " but expression is of type %s", - targetName, - format_type_be(targetTypeId), - format_type_be(exprType(rhs))), - errhint("You will need to rewrite or cast the expression."), - parser_errposition(pstate, location))); - else + result = coerce_to_target_type(pstate, + rhs, exprType(rhs), + targetTypeId, targetTypMod, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + + if (result == NULL) ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("subfield \"%s\" is of type %s" @@ -900,29 +894,42 @@ transformAssignmentSubscripts(ParseState *pstate, Node *rhs, int location) { - Node *result; - Oid containerType; - int32 containerTypMod; - Oid elementTypeId; - Oid typeNeeded; - Oid collationNeeded; + Node *result; + Oid containerType; + int32 containerTypMod; + Oid collationNeeded; + SubscriptingRef *sbsref; + SubscriptRoutines *sbsroutines; Assert(subscripts != NIL); /* Identify the actual array type and element type involved */ containerType = targetTypeId; containerTypMod = targetTypMod; - elementTypeId = transformContainerType(&containerType, &containerTypMod); - /* Identify type that RHS must provide */ - typeNeeded = isSlice ? containerType : elementTypeId; + /* process subscripts */ + sbsref = transformContainerSubscripts(pstate, + basenode, + containerType, + exprType(rhs), + containerTypMod, + subscripts, + rhs); + + sbsroutines = getSubscriptingRoutines(sbsref->refcontainertype); + + /* + * Let custom code provide necessary information about required types: + * refelemtype and refassgntype + */ + sbsref = sbsroutines->prepare(rhs != NULL, sbsref); /* * container normally has same collation as elements, but there's an * exception: we might be subscripting a domain over a container type. In * that case use collation of the base type. */ - if (containerType == targetTypeId) + if (sbsref->refcontainertype == containerType) collationNeeded = targetCollation; else collationNeeded = get_typcollation(containerType); @@ -932,25 +939,22 @@ transformAssignmentSubscripts(ParseState *pstate, NULL, targetName, true, - typeNeeded, - containerTypMod, + sbsref->refassgntype, + sbsref->reftypmod, collationNeeded, indirection, next_indirection, rhs, location); - /* process subscripts */ - result = (Node *) transformContainerSubscripts(pstate, - basenode, - containerType, - elementTypeId, - containerTypMod, - subscripts, - rhs); + /* Provide fully prepared subscriptinng information for custom validation */ + sbsref->refassgnexpr = (Expr *) rhs; + sbsroutines->validate(rhs != NULL, sbsref, pstate); + + result = (Node *) sbsref; /* If target was a domain over container, need to coerce up to the domain */ - if (containerType != targetTypeId) + if (sbsref->refcontainertype != targetTypeId) { Oid resulttype = exprType(result); diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c index 5e63238f03..7d56a998ab 100644 --- a/src/backend/utils/adt/ruleutils.c +++ b/src/backend/utils/adt/ruleutils.c @@ -7980,17 +7980,18 @@ get_rule_expr(Node *node, deparse_context *context, if (need_parens) appendStringInfoChar(buf, ')'); - /* - * If there's a refassgnexpr, we want to print the node in the - * format "container[subscripts] := refassgnexpr". This is - * not legal SQL, so decompilation of INSERT or UPDATE - * statements should always use processIndirection as part of - * the statement-level syntax. We should only see this when - * EXPLAIN tries to print the targetlist of a plan resulting - * from such a statement. - */ - if (sbsref->refassgnexpr) + if (IsAssignment(sbsref)) { + /* + * If there's a refassgnexpr, we want to print the node in the + * format "container[subscripts] := refassgnexpr". This is not + * legal SQL, so decompilation of INSERT or UPDATE statements + * should always use processIndirection as part of the + * statement-level syntax. We should only see this when + * EXPLAIN tries to print the targetlist of a plan resulting + * from such a statement. + */ + Node *refassgnexpr; /* diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index 3da90cb72a..78a6006fac 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -3176,6 +3176,29 @@ get_range_collation(Oid rangeOid) return InvalidOid; } +/* + * get_typsubshandler + * + * Given the type OID, return the type's subscripting procedures, if any, + * through pointers in arguments. + */ +RegProcedure +get_typsubsprocs(Oid typid) +{ + HeapTuple tp; + + tp = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid)); + if (HeapTupleIsValid(tp)) + { + RegProcedure handler = ((Form_pg_type) GETSTRUCT(tp))->typsubshandler; + ReleaseSysCache(tp); + + return handler; + } + else + return InvalidOid; +} + /* ---------- PG_INDEX CACHE ---------- */ /* diff --git a/src/include/c.h b/src/include/c.h index 831c89f473..31ef7c8076 100644 --- a/src/include/c.h +++ b/src/include/c.h @@ -538,6 +538,8 @@ typedef struct int indx[MAXDIM]; } IntArray; +#define MAX_SUBSCRIPT_DEPTH 12 + /* ---------------- * Variable-length datatypes all share the 'struct varlena' header. * diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h index f972f941e0..7e2b8d6dc7 100644 --- a/src/include/catalog/pg_type.h +++ b/src/include/catalog/pg_type.h @@ -221,6 +221,12 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati */ Oid typcollation BKI_DEFAULT(0) BKI_LOOKUP(pg_collation); + /* + * Type specific subscripting logic. If typsubshandler is none, it means + * that this type doesn't support subscripting. + */ + regproc typsubshandler BKI_DEFAULT(-) BKI_LOOKUP(pg_proc); + #ifdef CATALOG_VARLEN /* variable-length fields start here */ /* @@ -338,7 +344,8 @@ extern ObjectAddress TypeCreate(Oid newTypeOid, int32 typeMod, int32 typNDims, bool typeNotNull, - Oid typeCollation); + Oid typeCollation, + Oid subscriptingParseProcedure); extern void GenerateTypeDependencies(Oid typeObjectId, Form_pg_type typeForm, diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h index 8bbf6621da..846b834f2d 100644 --- a/src/include/executor/execExpr.h +++ b/src/include/executor/execExpr.h @@ -19,7 +19,7 @@ /* forward references to avoid circularity */ struct ExprEvalStep; -struct SubscriptingRefState; +struct SubscriptRoutines; /* Bits in ExprState->flags (see also execnodes.h for public flag bits): */ /* expression's interpreter has been initialized */ @@ -650,13 +650,13 @@ typedef struct SubscriptingRefState /* numupper and upperprovided[] are filled at compile time */ /* at runtime, extracted subscript datums get stored in upperindex[] */ int numupper; - bool upperprovided[MAXDIM]; - int upperindex[MAXDIM]; + bool upperprovided[MAX_SUBSCRIPT_DEPTH]; + Datum upperindex[MAX_SUBSCRIPT_DEPTH]; /* similarly for lower indexes, if any */ int numlower; - bool lowerprovided[MAXDIM]; - int lowerindex[MAXDIM]; + bool lowerprovided[MAX_SUBSCRIPT_DEPTH]; + Datum lowerindex[MAX_SUBSCRIPT_DEPTH]; /* subscript expressions get evaluated into here */ Datum subscriptvalue; @@ -669,6 +669,9 @@ typedef struct SubscriptingRefState /* if we have a nested assignment, SBSREF_OLD puts old value here */ Datum prevvalue; bool prevnull; + + bool resnull; + struct SubscriptRoutines *sbsroutines; } SubscriptingRefState; diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h index d73be2ad46..5991f437cd 100644 --- a/src/include/nodes/primnodes.h +++ b/src/include/nodes/primnodes.h @@ -417,13 +417,17 @@ typedef struct SubscriptingRef Expr xpr; Oid refcontainertype; /* type of the container proper */ Oid refelemtype; /* type of the container elements */ + Oid refassgntype; /* type of assignment expr that is required */ int32 reftypmod; /* typmod of the container (and elements too) */ Oid refcollid; /* OID of collation, or InvalidOid if none */ + Oid refnestedfunc; /* OID of type-specific function to handle nested assignment */ List *refupperindexpr; /* expressions that evaluate to upper * container indexes */ List *reflowerindexpr; /* expressions that evaluate to lower * container indexes, or NIL for single * container element */ + List *refindexprslice; /* whether or not related indexpr from + * reflowerindexpr is a slice */ Expr *refexpr; /* the expression that evaluates to a * container value */ @@ -431,6 +435,8 @@ typedef struct SubscriptingRef * fetch */ } SubscriptingRef; +#define IsAssignment(expr) ( ((SubscriptingRef*) expr)->refassgnexpr != NULL ) + /* * CoercionContext - distinguishes the allowed set of type casts * diff --git a/src/include/nodes/subscripting.h b/src/include/nodes/subscripting.h new file mode 100644 index 0000000000..1800d5ecf5 --- /dev/null +++ b/src/include/nodes/subscripting.h @@ -0,0 +1,42 @@ +/*------------------------------------------------------------------------- + * + * subscripting.h + * API for generic type subscripting + * + * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/nodes/subscripting.h + * + *------------------------------------------------------------------------- + */ +#ifndef SUBSCRIPTING_H +#define SUBSCRIPTING_H + +#include "parser/parse_node.h" +#include "nodes/primnodes.h" + +struct ParseState; +struct SubscriptingRefState; + +/* Callback function signatures --- see xsubscripting.sgml for more info. */ +typedef SubscriptingRef * (*SubscriptingPrepare) (bool isAssignment, SubscriptingRef *sbsef); + +typedef SubscriptingRef * (*SubscriptingValidate) (bool isAssignment, SubscriptingRef *sbsef, + struct ParseState *pstate); + +typedef Datum (*SubscriptingFetch) (Datum source, struct SubscriptingRefState *sbsrefstate); + +typedef Datum (*SubscriptingAssign) (Datum source, struct SubscriptingRefState *sbrsefstate); + +typedef struct SubscriptRoutines +{ + SubscriptingPrepare prepare; + SubscriptingValidate validate; + SubscriptingFetch fetch; + SubscriptingAssign assign; + +} SubscriptRoutines; + + +#endif /* SUBSCRIPTING_H */ diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h index d25819aa28..2aa64e788f 100644 --- a/src/include/parser/parse_node.h +++ b/src/include/parser/parse_node.h @@ -15,6 +15,7 @@ #define PARSE_NODE_H #include "nodes/parsenodes.h" +#include "nodes/subscripting.h" #include "utils/queryenvironment.h" #include "utils/relcache.h" @@ -313,7 +314,7 @@ extern void setup_parser_errposition_callback(ParseCallbackState *pcbstate, ParseState *pstate, int location); extern void cancel_parser_errposition_callback(ParseCallbackState *pcbstate); -extern Oid transformContainerType(Oid *containerType, int32 *containerTypmod); +extern void transformContainerType(Oid *containerType, int32 *containerTypmod); extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate, Node *containerBase, @@ -322,6 +323,7 @@ extern SubscriptingRef *transformContainerSubscripts(ParseState *pstate, int32 containerTypMod, List *indirection, Node *assignFrom); +extern SubscriptRoutines* getSubscriptingRoutines(Oid containerType); extern Const *make_const(ParseState *pstate, Value *value, int location); #endif /* PARSE_NODE_H */ diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index f132d39458..ff8f187b38 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -180,6 +180,7 @@ extern char *get_namespace_name(Oid nspid); extern char *get_namespace_name_or_temp(Oid nspid); extern Oid get_range_subtype(Oid rangeOid); extern Oid get_range_collation(Oid rangeOid); +extern RegProcedure get_typsubsprocs(Oid typid); extern Oid get_index_column_opclass(Oid index_oid, int attno); #define type_is_array(typid) (get_element_type(typid) != InvalidOid) -- 2.21.0 --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0002-Subscripting-for-array.patch" From be61cb04b13f5f06b1efda08f36f1e8eee5d90a4 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Fri, 1 Feb 2019 11:38:52 +0100 Subject: [PATCH v30 2/6] Subscripting for array Subscripting implementation for array data types. It includes all array specific parts, that were removed from the generalized code. Note, that for some array-like data types it's not necessary to assign array_subscript_handler explicitely, since it's done in Catalog.pm Reviewed-by: Tom Lane, Arthur Zakirov --- src/backend/catalog/Catalog.pm | 1 + src/backend/catalog/heap.c | 2 +- src/backend/commands/typecmds.c | 8 +- src/backend/executor/execExprInterp.c | 15 +- src/backend/nodes/nodeFuncs.c | 2 +- src/backend/parser/parse_node.c | 11 - src/backend/parser/parse_target.c | 4 +- src/backend/utils/adt/arrayfuncs.c | 289 ++++++++++++++++++++++++++ src/include/catalog/pg_proc.dat | 7 + src/include/catalog/pg_type.dat | 30 ++- src/test/regress/expected/arrays.out | 12 +- src/test/regress/sql/arrays.sql | 4 +- 12 files changed, 346 insertions(+), 39 deletions(-) diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Catalog.pm index dd39a086ce..b4dfa26518 100644 --- a/src/backend/catalog/Catalog.pm +++ b/src/backend/catalog/Catalog.pm @@ -384,6 +384,7 @@ sub GenerateArrayTypes # Arrays require INT alignment, unless the element type requires # DOUBLE alignment. $array_type{typalign} = $elem_type->{typalign} eq 'd' ? 'd' : 'i'; + $array_type{typsubshandler} = 'array_subscript_handler'; # Fill in the rest of the array entry's fields. foreach my $column (@$pgtype_schema) diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index db746c99b6..2fb34b30c3 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -1341,7 +1341,7 @@ heap_create_with_catalog(const char *relname, 0, /* array dimensions for typBaseType */ false, /* Type NOT NULL */ InvalidOid, /* rowtypes never have a collation */ - 0); /* array implementation */ + F_ARRAY_SUBSCRIPT_HANDLER); /* array implementation */ pfree(relarrayname); } diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c index 660e6df39a..16c9f6bd6b 100644 --- a/src/backend/commands/typecmds.c +++ b/src/backend/commands/typecmds.c @@ -690,7 +690,7 @@ DefineType(ParseState *pstate, List *names, List *parameters) 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ collation, /* type's collation */ - 0); + F_ARRAY_SUBSCRIPT_HANDLER); pfree(array_type); @@ -1131,7 +1131,7 @@ DefineDomain(CreateDomainStmt *stmt) 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ domaincoll, /* type's collation */ - 0); /* array subscripting implementation */ + F_ARRAY_SUBSCRIPT_HANDLER); /* array subscripting implementation */ pfree(domainArrayName); @@ -1288,7 +1288,7 @@ DefineEnum(CreateEnumStmt *stmt) 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ InvalidOid, /* type's collation */ - 0); /* array subscripting implementation */ + F_ARRAY_SUBSCRIPT_HANDLER); /* array subscripting implementation */ pfree(enumArrayName); @@ -1620,7 +1620,7 @@ DefineRange(CreateRangeStmt *stmt) 0, /* Array dimensions of typbasetype */ false, /* Type NOT NULL */ InvalidOid, /* typcollation */ - 0); /* array subscripting implementation */ + F_ARRAY_SUBSCRIPT_HANDLER); /* array subscripting implementation */ pfree(rangeArrayName); diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c index a945cad9d3..0a103bb403 100644 --- a/src/backend/executor/execExprInterp.c +++ b/src/backend/executor/execExprInterp.c @@ -3143,7 +3143,7 @@ ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) if (sbsrefstate->isassignment) ereport(ERROR, (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED), - errmsg("array subscript in assignment must not be null"))); + errmsg("subscript in assignment must not be null"))); *op->resnull = true; return false; } @@ -3215,9 +3215,20 @@ ExecEvalSubscriptingRefOld(ExprState *state, ExprEvalStep *op) void ExecEvalSubscriptingRefAssign(ExprState *state, ExprEvalStep *op) { - SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; + SubscriptingRefState *sbsrefstate = op->d.sbsref.state; SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; + /* + * For an assignment to a fixed-length container type, both the original + * container and the value to be assigned into it must be non-NULL, else we + * punt and return the original container. + */ + if (sbsrefstate->refattrlength > 0) + { + if (*op->resnull || sbsrefstate->replacenull) + return; + } + sbsrefstate->resnull = *op->resnull; *op->resvalue = sbsroutines->assign(*op->resvalue, sbsrefstate); *op->resnull = sbsrefstate->resnull; diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c index d85ca9f7c5..7e995a66d0 100644 --- a/src/backend/nodes/nodeFuncs.c +++ b/src/backend/nodes/nodeFuncs.c @@ -70,7 +70,7 @@ exprType(const Node *expr) const SubscriptingRef *sbsref = (const SubscriptingRef *) expr; /* slice and/or store operations yield the container type */ - if (sbsref->reflowerindexpr || sbsref->refassgnexpr) + if (IsAssignment(sbsref) || sbsref->reflowerindexpr) type = sbsref->refcontainertype; else type = sbsref->refelemtype; diff --git a/src/backend/parser/parse_node.c b/src/backend/parser/parse_node.c index 12dde390d3..3710d01aa2 100644 --- a/src/backend/parser/parse_node.c +++ b/src/backend/parser/parse_node.c @@ -309,17 +309,6 @@ transformContainerSubscripts(ParseState *pstate, lowerIndexpr = lappend(lowerIndexpr, subexpr); indexprSlice = lappend(indexprSlice, ai); } - else - Assert(ai->lidx == NULL && !ai->is_slice); - - if (ai->uidx) - subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); - else - { - /* Slice with omitted upper bound, put NULL into the list */ - Assert(isSlice && ai->is_slice); - subexpr = NULL; - } subexpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); upperIndexpr = lappend(upperIndexpr, subexpr); } diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c index a5e734837f..3ca0791500 100644 --- a/src/backend/parser/parse_target.c +++ b/src/backend/parser/parse_target.c @@ -685,7 +685,7 @@ transformAssignmentIndirection(ParseState *pstate, Node *rhs, int location) { - Node *result = NULL; + Node *result; List *subscripts = NIL; bool isSlice = false; ListCell *i; @@ -873,6 +873,8 @@ transformAssignmentIndirection(ParseState *pstate, errhint("You will need to rewrite or cast the expression."), parser_errposition(pstate, location))); } + else + result = rhs; return result; } diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 7a4a5aaa86..01bc3e8ac5 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -24,14 +24,20 @@ #include "nodes/nodeFuncs.h" #include "nodes/supportnodes.h" #include "optimizer/optimizer.h" +#include "nodes/makefuncs.h" +#include "executor/execExpr.h" #include "utils/array.h" #include "utils/arrayaccess.h" #include "utils/builtins.h" #include "utils/datum.h" +#include "utils/fmgroids.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/selfuncs.h" +#include "utils/syscache.h" #include "utils/typcache.h" +#include "parser/parse_node.h" +#include "parser/parse_coerce.h" /* @@ -158,7 +164,14 @@ static int width_bucket_array_variable(Datum operand, ArrayType *thresholds, Oid collation, TypeCacheEntry *typentry); +static SubscriptingRef *array_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref); +static SubscriptingRef *array_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate); +static Datum array_subscript_fetch(Datum containerSource, + SubscriptingRefState *sbstate); +static Datum array_subscript_assign(Datum containerSource, + SubscriptingRefState *sbstate); /* * array_in : @@ -6598,3 +6611,279 @@ width_bucket_array_variable(Datum operand, return left; } + +/* + * Perform an actual data extraction or modification for the array + * subscripting. As a result the extracted Datum or the modified containers + * value will be returned. + */ +Datum +array_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) +{ + bool is_slice = (sbstate->numlower != 0); + IntArray u_index, l_index; + bool eisnull = sbstate->resnull; + int i = 0; + + if (sbstate->refelemlength == 0) + { + /* do one-time catalog lookups for type info */ + get_typlenbyvalalign(sbstate->refelemtype, + &sbstate->refelemlength, + &sbstate->refelembyval, + &sbstate->refelemalign); + } + + for(i = 0; i < sbstate->numupper; i++) + u_index.indx[i] = DatumGetInt32(sbstate->upperindex[i]); + + if (is_slice) + { + for(i = 0; i < sbstate->numlower; i++) + l_index.indx[i] = DatumGetInt32(sbstate->lowerindex[i]); + } + + /* + * For assignment to varlena arrays, we handle a NULL original array + * by substituting an empty (zero-dimensional) array; insertion of the + * new element will result in a singleton array value. It does not + * matter whether the new element is NULL. + */ + if (eisnull) + { + containerSource = PointerGetDatum(construct_empty_array(sbstate->refelemtype)); + sbstate->resnull = false; + eisnull = false; + } + + if (!is_slice) + return array_set_element(containerSource, sbstate->numupper, + u_index.indx, + sbstate->replacevalue, + sbstate->replacenull, + sbstate->refattrlength, + sbstate->refelemlength, + sbstate->refelembyval, + sbstate->refelemalign); + else + return array_set_slice(containerSource, sbstate->numupper, + u_index.indx, l_index.indx, + sbstate->upperprovided, + sbstate->lowerprovided, + sbstate->replacevalue, + sbstate->replacenull, + sbstate->refattrlength, + sbstate->refelemlength, + sbstate->refelembyval, + sbstate->refelemalign); +} + +Datum +array_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) +{ + bool is_slice = (sbstate->numlower != 0); + IntArray u_index, l_index; + int i = 0; + + if (sbstate->refelemlength == 0) + { + /* do one-time catalog lookups for type info */ + get_typlenbyvalalign(sbstate->refelemtype, + &sbstate->refelemlength, + &sbstate->refelembyval, + &sbstate->refelemalign); + } + + for(i = 0; i < sbstate->numupper; i++) + u_index.indx[i] = DatumGetInt32(sbstate->upperindex[i]); + + if (is_slice) + { + for(i = 0; i < sbstate->numlower; i++) + l_index.indx[i] = DatumGetInt32(sbstate->lowerindex[i]); + } + + if (!is_slice) + return array_get_element(containerSource, sbstate->numupper, + u_index.indx, + sbstate->refattrlength, + sbstate->refelemlength, + sbstate->refelembyval, + sbstate->refelemalign, + &sbstate->resnull); + else + return array_get_slice(containerSource, sbstate->numupper, + u_index.indx, l_index.indx, + sbstate->upperprovided, + sbstate->lowerprovided, + sbstate->refattrlength, + sbstate->refelemlength, + sbstate->refelembyval, + sbstate->refelemalign); +} + +/* + * Handle array-type subscripting logic. + */ +Datum +array_subscript_handler(PG_FUNCTION_ARGS) +{ + SubscriptRoutines *sbsroutines = (SubscriptRoutines *) + palloc(sizeof(SubscriptRoutines)); + + sbsroutines->prepare = array_subscript_prepare; + sbsroutines->validate = array_subscript_validate; + sbsroutines->fetch = array_subscript_fetch; + sbsroutines->assign = array_subscript_assign; + + PG_RETURN_POINTER(sbsroutines); +} + +SubscriptingRef * +array_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref) +{ + Oid array_type = sbsref->refcontainertype; + HeapTuple type_tuple_container; + Form_pg_type type_struct_container; + bool is_slice = sbsref->reflowerindexpr != NIL; + + /* Get the type tuple for the container */ + type_tuple_container = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_type)); + if (!HeapTupleIsValid(type_tuple_container)) + elog(ERROR, "cache lookup failed for type %u", array_type); + type_struct_container = (Form_pg_type) GETSTRUCT(type_tuple_container); + + /* needn't check typisdefined since this will fail anyway */ + sbsref->refelemtype = type_struct_container->typelem; + + /* Identify type that RHS must provide */ + if (isAssignment) + sbsref->refassgntype = is_slice ? sbsref->refcontainertype : sbsref->refelemtype; + + ReleaseSysCache(type_tuple_container); + + return sbsref; +} + +SubscriptingRef * +array_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate) +{ + bool is_slice = sbsref->reflowerindexpr != NIL; + Oid typeneeded = InvalidOid, + typesource = InvalidOid; + Node *new_from; + Node *subexpr; + List *upperIndexpr = NIL; + List *lowerIndexpr = NIL; + ListCell *u, *l, *s; + + foreach(u, sbsref->refupperindexpr) + { + subexpr = (Node *) lfirst(u); + + if (subexpr == NULL) + { + upperIndexpr = lappend(upperIndexpr, subexpr); + continue; + } + + subexpr = coerce_to_target_type(pstate, + subexpr, exprType(subexpr), + INT4OID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("array subscript must have type integer"), + parser_errposition(pstate, exprLocation(subexpr)))); + + upperIndexpr = lappend(upperIndexpr, subexpr); + } + + sbsref->refupperindexpr = upperIndexpr; + + forboth(l, sbsref->reflowerindexpr, s, sbsref->refindexprslice) + { + A_Indices *ai = (A_Indices *) lfirst(s); + subexpr = (Node *) lfirst(l); + + if (subexpr == NULL && !ai->is_slice) + { + /* Make a constant 1 */ + subexpr = (Node *) makeConst(INT4OID, + -1, + InvalidOid, + sizeof(int32), + Int32GetDatum(1), + false, + true); /* pass by value */ + } + + if (subexpr == NULL) + { + lowerIndexpr = lappend(lowerIndexpr, subexpr); + continue; + } + + subexpr = coerce_to_target_type(pstate, + subexpr, exprType(subexpr), + INT4OID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("array subscript must have type integer"), + parser_errposition(pstate, exprLocation(subexpr)))); + + lowerIndexpr = lappend(lowerIndexpr, subexpr); + } + + sbsref->reflowerindexpr = lowerIndexpr; + + if (isAssignment) + { + SubscriptingRef *assignRef = (SubscriptingRef *) sbsref; + Node *assignExpr = (Node *) assignRef->refassgnexpr; + + typesource = exprType(assignExpr); + typeneeded = is_slice ? sbsref->refcontainertype : sbsref->refelemtype; + new_from = coerce_to_target_type(pstate, + assignExpr, typesource, + typeneeded, sbsref->reftypmod, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (new_from == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("array assignment requires type %s" + " but expression is of type %s", + format_type_be(sbsref->refelemtype), + format_type_be(typesource)), + errhint("You will need to rewrite or cast the expression."), + parser_errposition(pstate, exprLocation(assignExpr)))); + assignRef->refassgnexpr = (Expr *) new_from; + } + + sbsref->refnestedfunc = F_ARRAY_SUBSCRIPT_HANDLER; + + /* Verify subscript list lengths are within limit */ + if (list_length(sbsref->refupperindexpr) > MAXDIM) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", + list_length(sbsref->refupperindexpr), MAXDIM))); + + if (list_length(sbsref->reflowerindexpr) > MAXDIM) + ereport(ERROR, + (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)", + list_length(sbsref->reflowerindexpr), MAXDIM))); + + return sbsref; +} diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 07a86c7b7b..6de2dd29f1 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10707,6 +10707,13 @@ proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version}', prosrc => 'pg_control_init' }, +{ oid => '4192', + descr => 'Array subscripting logic', + proname => 'array_subscript_handler', + prorettype => 'internal', + proargtypes => 'internal', + prosrc => 'array_subscript_handler' }, + # collation management functions { oid => '3445', descr => 'import collations from operating system', proname => 'pg_import_system_collations', procost => '100', diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index 4cf2b9df7b..f868a6d52e 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -54,7 +54,8 @@ typname => 'name', typlen => 'NAMEDATALEN', typbyval => 'f', typcategory => 'S', typelem => 'char', typinput => 'namein', typoutput => 'nameout', typreceive => 'namerecv', typsend => 'namesend', - typalign => 'c', typcollation => 'C' }, + typalign => 'c', typcollation => 'C', + typsubshandler => 'array_subscript_handler' }, { oid => '20', array_type_oid => '1016', descr => '~18 digit integer, 8-byte storage', typname => 'int8', typlen => '8', typbyval => 'FLOAT8PASSBYVAL', @@ -70,7 +71,7 @@ typname => 'int2vector', typlen => '-1', typbyval => 'f', typcategory => 'A', typelem => 'int2', typinput => 'int2vectorin', typoutput => 'int2vectorout', typreceive => 'int2vectorrecv', typsend => 'int2vectorsend', - typalign => 'i' }, + typalign => 'i', typsubshandler => 'array_subscript_handler' }, { oid => '23', array_type_oid => '1007', descr => '-2 billion to 2 billion integer, 4-byte storage', typname => 'int4', typlen => '4', typbyval => 't', typcategory => 'N', @@ -109,7 +110,8 @@ descr => 'array of oids, used in system tables', typname => 'oidvector', typlen => '-1', typbyval => 'f', typcategory => 'A', typelem => 'oid', typinput => 'oidvectorin', typoutput => 'oidvectorout', - typreceive => 'oidvectorrecv', typsend => 'oidvectorsend', typalign => 'i' }, + typreceive => 'oidvectorrecv', typsend => 'oidvectorsend', typalign => 'i', + typsubshandler => 'array_subscript_handler' }, # hand-built rowtype entries for bootstrapped catalogs # NB: OIDs assigned here must match the BKI_ROWTYPE_OID declarations @@ -184,32 +186,37 @@ descr => 'geometric point \'(x, y)\'', typname => 'point', typlen => '16', typbyval => 'f', typcategory => 'G', typelem => 'float8', typinput => 'point_in', typoutput => 'point_out', - typreceive => 'point_recv', typsend => 'point_send', typalign => 'd' }, + typreceive => 'point_recv', typsend => 'point_send', typalign => 'd', + typsubshandler => 'array_subscript_handler' }, { oid => '601', array_type_oid => '1018', descr => 'geometric line segment \'(pt1,pt2)\'', typname => 'lseg', typlen => '32', typbyval => 'f', typcategory => 'G', typelem => 'point', typinput => 'lseg_in', typoutput => 'lseg_out', - typreceive => 'lseg_recv', typsend => 'lseg_send', typalign => 'd' }, + typreceive => 'lseg_recv', typsend => 'lseg_send', typalign => 'd', + typsubshandler => 'array_subscript_handler' }, { oid => '602', array_type_oid => '1019', descr => 'geometric path \'(pt1,...)\'', typname => 'path', typlen => '-1', typbyval => 'f', typcategory => 'G', typinput => 'path_in', typoutput => 'path_out', typreceive => 'path_recv', - typsend => 'path_send', typalign => 'd', typstorage => 'x' }, + typsend => 'path_send', typalign => 'd', typstorage => 'x', + typsubshandler => 'array_subscript_handler' }, { oid => '603', array_type_oid => '1020', descr => 'geometric box \'(lower left,upper right)\'', typname => 'box', typlen => '32', typbyval => 'f', typcategory => 'G', typdelim => ';', typelem => 'point', typinput => 'box_in', typoutput => 'box_out', typreceive => 'box_recv', typsend => 'box_send', - typalign => 'd' }, + typalign => 'd', typsubshandler => 'array_subscript_handler' }, { oid => '604', array_type_oid => '1027', descr => 'geometric polygon \'(pt1,...)\'', typname => 'polygon', typlen => '-1', typbyval => 'f', typcategory => 'G', typinput => 'poly_in', typoutput => 'poly_out', typreceive => 'poly_recv', - typsend => 'poly_send', typalign => 'd', typstorage => 'x' }, + typsend => 'poly_send', typalign => 'd', typstorage => 'x', + typsubshandler => 'array_subscript_handler' }, { oid => '628', array_type_oid => '629', descr => 'geometric line', typname => 'line', typlen => '24', typbyval => 'f', typcategory => 'G', typelem => 'float8', typinput => 'line_in', typoutput => 'line_out', - typreceive => 'line_recv', typsend => 'line_send', typalign => 'd' }, + typreceive => 'line_recv', typsend => 'line_send', typalign => 'd', + typsubshandler => 'array_subscript_handler' }, # OIDS 700 - 799 @@ -268,7 +275,7 @@ { oid => '1033', array_type_oid => '1034', descr => 'access control list', typname => 'aclitem', typlen => '12', typbyval => 'f', typcategory => 'U', typinput => 'aclitemin', typoutput => 'aclitemout', typreceive => '-', - typsend => '-', typalign => 'i' }, + typsend => '-', typalign => 'i', typsubshandler => 'array_subscript_handler' }, { oid => '1042', array_type_oid => '1014', descr => 'char(length), blank-padded string, fixed storage length', typname => 'bpchar', typlen => '-1', typbyval => 'f', typcategory => 'S', @@ -307,7 +314,8 @@ typcategory => 'D', typispreferred => 't', typinput => 'timestamptz_in', typoutput => 'timestamptz_out', typreceive => 'timestamptz_recv', typsend => 'timestamptz_send', typmodin => 'timestamptztypmodin', - typmodout => 'timestamptztypmodout', typalign => 'd' }, + typmodout => 'timestamptztypmodout', typalign => 'd', + typsubshandler => 'array_subscript_handler' }, { oid => '1186', array_type_oid => '1187', descr => '@ , time interval', typname => 'interval', typlen => '16', typbyval => 'f', typcategory => 'T', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index c730563f03..f54b73c9da 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -190,9 +190,9 @@ select ('[0:2][0:2]={{1,2,3},{4,5,6},{7,8,9}}'::int[])[1:2][2]; -- -- check subscription corner cases -- --- More subscripts than MAXDIMS(6) -SELECT ('{}'::int[])[1][2][3][4][5][6][7]; -ERROR: number of array dimensions (7) exceeds the maximum allowed (6) +-- More subscripts than MAXDIMS(12) +SELECT ('{}'::int[])[1][2][3][4][5][6][7][8][9][10][11][12][13]; +ERROR: number of array dimensions (13) exceeds the maximum allowed (6) -- NULL index yields NULL when selecting SELECT ('{{{1},{2},{3}},{{4},{5},{6}}}'::int[])[1][NULL][1]; int4 @@ -216,15 +216,15 @@ SELECT ('{{{1},{2},{3}},{{4},{5},{6}}}'::int[])[1][1:NULL][1]; UPDATE arrtest SET c[NULL] = '{"can''t assign"}' WHERE array_dims(c) is not null; -ERROR: array subscript in assignment must not be null +ERROR: subscript in assignment must not be null UPDATE arrtest SET c[NULL:1] = '{"can''t assign"}' WHERE array_dims(c) is not null; -ERROR: array subscript in assignment must not be null +ERROR: subscript in assignment must not be null UPDATE arrtest SET c[1:NULL] = '{"can''t assign"}' WHERE array_dims(c) is not null; -ERROR: array subscript in assignment must not be null +ERROR: subscript in assignment must not be null -- test slices with empty lower and/or upper index CREATE TEMP TABLE arrtest_s ( a int2[], diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 25dd4e2c6d..fb7a319118 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -106,8 +106,8 @@ select ('[0:2][0:2]={{1,2,3},{4,5,6},{7,8,9}}'::int[])[1:2][2]; -- -- check subscription corner cases -- --- More subscripts than MAXDIMS(6) -SELECT ('{}'::int[])[1][2][3][4][5][6][7]; +-- More subscripts than MAXDIMS(12) +SELECT ('{}'::int[])[1][2][3][4][5][6][7][8][9][10][11][12][13]; -- NULL index yields NULL when selecting SELECT ('{{{1},{2},{3}},{{4},{5},{6}}}'::int[])[1][NULL][1]; SELECT ('{{{1},{2},{3}},{{4},{5},{6}}}'::int[])[1][NULL:1][1]; -- 2.21.0 --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0003-Subscripting-for-jsonb.patch" From 0c43ba6ceff7e8c19163768ee0f2665856199ce4 Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Fri, 1 Feb 2019 11:41:45 +0100 Subject: [PATCH v30 3/6] Subscripting for jsonb Subscripting implementation for jsonb. For the sake of code reuse, some parts of jsonb functionality were rearranged to allow use the same functions for jsonb_set and assign subscripting operation. Reviewed-by: Tom Lane, Arthur Zakirov --- src/backend/utils/adt/jsonb.c | 27 ++- src/backend/utils/adt/jsonb_util.c | 76 ++++++- src/backend/utils/adt/jsonfuncs.c | 325 ++++++++++++++++++++-------- src/include/catalog/pg_proc.dat | 8 + src/include/catalog/pg_type.dat | 3 +- src/include/utils/jsonb.h | 2 + src/test/regress/expected/jsonb.out | 233 +++++++++++++++++++- src/test/regress/sql/jsonb.sql | 68 +++++- 8 files changed, 632 insertions(+), 110 deletions(-) diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c index b961d29472..ad364b4a0e 100644 --- a/src/backend/utils/adt/jsonb.c +++ b/src/backend/utils/adt/jsonb.c @@ -1134,23 +1134,34 @@ to_jsonb(PG_FUNCTION_ARGS) { Datum val = PG_GETARG_DATUM(0); Oid val_type = get_fn_expr_argtype(fcinfo->flinfo, 0); - JsonbInState result; - JsonbTypeCategory tcategory; - Oid outfuncoid; + JsonbValue *res = to_jsonb_worker(val, val_type, false); + PG_RETURN_POINTER(JsonbValueToJsonb(res)); +} - if (val_type == InvalidOid) +/* + * Do the actual conversion to jsonb for to_jsonb function. This logic is + * separated because it can be useful not only in here (e.g. we use it in + * jsonb subscripting) + */ +JsonbValue * +to_jsonb_worker(Datum source, Oid source_type, bool is_null) +{ + JsonbInState result; + JsonbTypeCategory tcategory; + Oid outfuncoid; + + if (source_type == InvalidOid) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("could not determine input data type"))); - jsonb_categorize_type(val_type, + jsonb_categorize_type(source_type, &tcategory, &outfuncoid); memset(&result, 0, sizeof(JsonbInState)); - datum_to_jsonb(val, false, &result, tcategory, outfuncoid, false); - - PG_RETURN_POINTER(JsonbValueToJsonb(result.res)); + datum_to_jsonb(source, is_null, &result, tcategory, outfuncoid, false); + return result.res; } /* diff --git a/src/backend/utils/adt/jsonb_util.c b/src/backend/utils/adt/jsonb_util.c index 04b70c805b..adcf16acf1 100644 --- a/src/backend/utils/adt/jsonb_util.c +++ b/src/backend/utils/adt/jsonb_util.c @@ -68,18 +68,29 @@ static JsonbValue *pushJsonbValueScalar(JsonbParseState **pstate, JsonbIteratorToken seq, JsonbValue *scalarVal); +JsonbValue * +JsonbToJsonbValue(Jsonb *jsonb) +{ + JsonbValue *val = (JsonbValue *) palloc(sizeof(JsonbValue)); + + val->type = jbvBinary; + val->val.binary.data = &jsonb->root; + val->val.binary.len = VARSIZE(jsonb) - VARHDRSZ; + + return val; +} + /* * Turn an in-memory JsonbValue into a Jsonb for on-disk storage. * - * There isn't a JsonbToJsonbValue(), because generally we find it more - * convenient to directly iterate through the Jsonb representation and only - * really convert nested scalar values. JsonbIteratorNext() does this, so that - * clients of the iteration code don't have to directly deal with the binary - * representation (JsonbDeepContains() is a notable exception, although all - * exceptions are internal to this module). In general, functions that accept - * a JsonbValue argument are concerned with the manipulation of scalar values, - * or simple containers of scalar values, where it would be inconvenient to - * deal with a great amount of other state. + * Generally we find it more convenient to directly iterate through the Jsonb + * representation and only really convert nested scalar values. + * JsonbIteratorNext() does this, so that clients of the iteration code don't + * have to directly deal with the binary representation (JsonbDeepContains() is + * a notable exception, although all exceptions are internal to this module). + * In general, functions that accept a JsonbValue argument are concerned with + * the manipulation of scalar values, or simple containers of scalar values, + * where it would be inconvenient to deal with a great amount of other state. */ Jsonb * JsonbValueToJsonb(JsonbValue *val) @@ -563,6 +574,30 @@ pushJsonbValue(JsonbParseState **pstate, JsonbIteratorToken seq, JsonbValue *res = NULL; JsonbValue v; JsonbIteratorToken tok; + int i; + + if (jbval && (seq == WJB_ELEM || seq == WJB_VALUE) && jbval->type == jbvObject) + { + pushJsonbValue(pstate, WJB_BEGIN_OBJECT, NULL); + for (i = 0; i < jbval->val.object.nPairs; i++) + { + pushJsonbValue(pstate, WJB_KEY, &jbval->val.object.pairs[i].key); + pushJsonbValue(pstate, WJB_VALUE, &jbval->val.object.pairs[i].value); + } + + return pushJsonbValue(pstate, WJB_END_OBJECT, NULL); + } + + if (jbval && (seq == WJB_ELEM || seq == WJB_VALUE) && jbval->type == jbvArray) + { + pushJsonbValue(pstate, WJB_BEGIN_ARRAY, NULL); + for (i = 0; i < jbval->val.array.nElems; i++) + { + pushJsonbValue(pstate, WJB_ELEM, &jbval->val.array.elems[i]); + } + + return pushJsonbValue(pstate, WJB_END_ARRAY, NULL); + } if (!jbval || (seq != WJB_ELEM && seq != WJB_VALUE) || jbval->type != jbvBinary) @@ -573,9 +608,30 @@ pushJsonbValue(JsonbParseState **pstate, JsonbIteratorToken seq, /* unpack the binary and add each piece to the pstate */ it = JsonbIteratorInit(jbval->val.binary.data); + + if ((jbval->val.binary.data->header & JB_FSCALAR) && *pstate) + { + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_BEGIN_ARRAY); + Assert(v.type == jbvArray && v.val.array.rawScalar); + + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_ELEM); + + res = pushJsonbValueScalar(pstate, seq, &v); + + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_END_ARRAY); + Assert(it == NULL); + + return res; + } + while ((tok = JsonbIteratorNext(&it, &v, false)) != WJB_DONE) res = pushJsonbValueScalar(pstate, tok, - tok < WJB_BEGIN_ARRAY ? &v : NULL); + tok < WJB_BEGIN_ARRAY || + (tok == WJB_BEGIN_ARRAY && + v.val.array.rawScalar) ? &v : NULL); return res; } diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c index f92861d8d2..538740aec9 100644 --- a/src/backend/utils/adt/jsonfuncs.c +++ b/src/backend/utils/adt/jsonfuncs.c @@ -21,12 +21,16 @@ #include "common/jsonapi.h" #include "fmgr.h" #include "funcapi.h" +#include "executor/execExpr.h" #include "lib/stringinfo.h" #include "mb/pg_wchar.h" #include "miscadmin.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_coerce.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/hsearch.h" +#include "utils/fmgroids.h" #include "utils/json.h" #include "utils/jsonb.h" #include "utils/jsonfuncs.h" @@ -460,18 +464,22 @@ static Datum populate_domain(DomainIOData *io, Oid typid, const char *colname, /* functions supporting jsonb_delete, jsonb_set and jsonb_concat */ static JsonbValue *IteratorConcat(JsonbIterator **it1, JsonbIterator **it2, JsonbParseState **state); +static Datum jsonb_set_element(Datum datum, Datum *path, int path_len, + Datum sourceData, Oid source_type, bool is_null); +static Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath, + bool *isnull, bool as_text); static JsonbValue *setPath(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, - JsonbParseState **st, int level, Jsonb *newval, + JsonbParseState **st, int level, JsonbValue *newval, int op_type); static void setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 npairs, int op_type); + JsonbValue *newval, uint32 npairs, int op_type); static void setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, - int level, Jsonb *newval, uint32 nelems, int op_type); -static void addJsonbToParseState(JsonbParseState **jbps, Jsonb *jb); + int level, + JsonbValue *newval, uint32 nelems, int op_type); /* function supporting iterate_json_values */ static void iterate_values_scalar(void *state, char *token, JsonTokenType tokentype); @@ -486,6 +494,15 @@ static void transform_string_values_object_field_start(void *state, char *fname, static void transform_string_values_array_element_start(void *state, bool isnull); static void transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype); +static SubscriptingRef *jsonb_subscript_prepare(bool isAssignment, + SubscriptingRef *sbsref); + +static SubscriptingRef *jsonb_subscript_validate(bool isAssignment, + SubscriptingRef *sbsref, + ParseState *pstate); +static Datum jsonb_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate); +static Datum jsonb_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate); + /* * pg_parse_json_or_ereport * @@ -1461,13 +1478,9 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); Datum *pathtext; bool *pathnulls; + bool isnull; int npath; - int i; - bool have_object = false, - have_array = false; - JsonbValue *jbvp = NULL; - JsonbValue jbvbuf; - JsonbContainer *container; + Datum res; /* * If the array contains any null elements, return NULL, on the grounds @@ -1482,9 +1495,28 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) deconstruct_array(path, TEXTOID, -1, false, TYPALIGN_INT, &pathtext, &pathnulls, &npath); - /* Identify whether we have object, array, or scalar at top-level */ - container = &jb->root; + res = jsonb_get_element(jb, pathtext, npath, &isnull, as_text); + if (isnull) + PG_RETURN_NULL(); + else + PG_RETURN_DATUM(res); +} + +static Datum +jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) +{ + Jsonb *res; + JsonbContainer *container = &jb->root; + JsonbValue *jbvp = NULL; + JsonbValue tv; + int i; + bool have_object = false, + have_array = false; + + *isnull = false; + + /* Identify whether we have object, array, or scalar at top-level */ if (JB_ROOT_IS_OBJECT(jb)) have_object = true; else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb)) @@ -1509,7 +1541,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) { if (as_text) { - PG_RETURN_TEXT_P(cstring_to_text(JsonbToCString(NULL, + return PointerGetDatum(cstring_to_text(JsonbToCString(NULL, container, VARSIZE(jb)))); } @@ -1525,22 +1557,25 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) if (have_object) { jbvp = getKeyJsonValueFromContainer(container, - VARDATA(pathtext[i]), - VARSIZE(pathtext[i]) - VARHDRSZ, - &jbvbuf); + VARDATA(path[i]), + VARSIZE(path[i]) - VARHDRSZ, + NULL); } else if (have_array) { long lindex; uint32 index; - char *indextext = TextDatumGetCString(pathtext[i]); + char *indextext = TextDatumGetCString(path[i]); char *endptr; errno = 0; lindex = strtol(indextext, &endptr, 10); if (endptr == indextext || *endptr != '\0' || errno != 0 || lindex > INT_MAX || lindex < INT_MIN) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } if (lindex >= 0) { @@ -1558,7 +1593,10 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) nelements = JsonContainerSize(container); if (-lindex > nelements) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } else index = nelements + lindex; } @@ -1568,11 +1606,15 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) else { /* scalar, extraction yields a null */ - PG_RETURN_NULL(); + *isnull = true; + return PointerGetDatum(NULL); } if (jbvp == NULL) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } else if (i == npath - 1) break; @@ -1594,9 +1636,12 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) if (as_text) { if (jbvp->type == jbvNull) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } - PG_RETURN_TEXT_P(JsonbValueAsText(jbvp)); + return PointerGetDatum(JsonbValueAsText(jbvp)); } else { @@ -1607,6 +1652,32 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) } } +Datum +jsonb_set_element(Datum jsonbdatum, Datum *path, int path_len, + Datum sourceData, Oid source_type, bool is_null) +{ + Jsonb *jb = DatumGetJsonbP(jsonbdatum); + JsonbValue *newval, + *res; + JsonbParseState *state = NULL; + JsonbIterator *it; + bool *path_nulls = palloc0(path_len * sizeof(bool)); + + newval = to_jsonb_worker(sourceData, source_type, is_null); + + if (newval->type == jbvArray && newval->val.array.rawScalar) + *newval = newval->val.array.elems[0]; + + it = JsonbIteratorInit(&jb->root); + + res = setPath(&it, path, path_nulls, path_len, &state, 0, + newval, JB_PATH_CREATE); + + pfree(path_nulls); + + PG_RETURN_JSONB_P(JsonbValueToJsonb(res)); +} + /* * Return the text representation of the given JsonbValue. */ @@ -4166,58 +4237,6 @@ jsonb_strip_nulls(PG_FUNCTION_ARGS) PG_RETURN_POINTER(JsonbValueToJsonb(res)); } -/* - * Add values from the jsonb to the parse state. - * - * If the parse state container is an object, the jsonb is pushed as - * a value, not a key. - * - * This needs to be done using an iterator because pushJsonbValue doesn't - * like getting jbvBinary values, so we can't just push jb as a whole. - */ -static void -addJsonbToParseState(JsonbParseState **jbps, Jsonb *jb) -{ - JsonbIterator *it; - JsonbValue *o = &(*jbps)->contVal; - JsonbValue v; - JsonbIteratorToken type; - - it = JsonbIteratorInit(&jb->root); - - Assert(o->type == jbvArray || o->type == jbvObject); - - if (JB_ROOT_IS_SCALAR(jb)) - { - (void) JsonbIteratorNext(&it, &v, false); /* skip array header */ - Assert(v.type == jbvArray); - (void) JsonbIteratorNext(&it, &v, false); /* fetch scalar value */ - - switch (o->type) - { - case jbvArray: - (void) pushJsonbValue(jbps, WJB_ELEM, &v); - break; - case jbvObject: - (void) pushJsonbValue(jbps, WJB_VALUE, &v); - break; - default: - elog(ERROR, "unexpected parent of nested structure"); - } - } - else - { - while ((type = JsonbIteratorNext(&it, &v, false)) != WJB_DONE) - { - if (type == WJB_KEY || type == WJB_VALUE || type == WJB_ELEM) - (void) pushJsonbValue(jbps, type, &v); - else - (void) pushJsonbValue(jbps, type, NULL); - } - } - -} - /* * SQL function jsonb_pretty (jsonb) * @@ -4489,7 +4508,8 @@ jsonb_set(PG_FUNCTION_ARGS) { Jsonb *in = PG_GETARG_JSONB_P(0); ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); - Jsonb *newval = PG_GETARG_JSONB_P(2); + Jsonb *newjsonb = PG_GETARG_JSONB_P(2); + JsonbValue *newval = JsonbToJsonbValue(newjsonb); bool create = PG_GETARG_BOOL(3); JsonbValue *res = NULL; Datum *path_elems; @@ -4646,7 +4666,8 @@ jsonb_insert(PG_FUNCTION_ARGS) { Jsonb *in = PG_GETARG_JSONB_P(0); ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); - Jsonb *newval = PG_GETARG_JSONB_P(2); + Jsonb *newjsonb = PG_GETARG_JSONB_P(2); + JsonbValue *newval = JsonbToJsonbValue(newjsonb); bool after = PG_GETARG_BOOL(3); JsonbValue *res = NULL; Datum *path_elems; @@ -4809,7 +4830,7 @@ IteratorConcat(JsonbIterator **it1, JsonbIterator **it2, static JsonbValue * setPath(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, - JsonbParseState **st, int level, Jsonb *newval, int op_type) + JsonbParseState **st, int level, JsonbValue *newval, int op_type) { JsonbValue v; JsonbIteratorToken r; @@ -4862,11 +4883,11 @@ setPath(JsonbIterator **it, Datum *path_elems, static void setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 npairs, int op_type) + JsonbValue *newval, uint32 npairs, int op_type) { - JsonbValue v; int i; - JsonbValue k; + JsonbValue k, + v; bool done = false; if (level >= path_len || path_nulls[level]) @@ -4883,7 +4904,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, newkey.val.string.val = VARDATA_ANY(path_elems[level]); (void) pushJsonbValue(st, WJB_KEY, &newkey); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } for (i = 0; i < npairs; i++) @@ -4914,7 +4935,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, if (!(op_type & JB_PATH_DELETE)) { (void) pushJsonbValue(st, WJB_KEY, &k); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } done = true; } @@ -4937,7 +4958,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, newkey.val.string.val = VARDATA_ANY(path_elems[level]); (void) pushJsonbValue(st, WJB_KEY, &newkey); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } (void) pushJsonbValue(st, r, &k); @@ -4969,7 +4990,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, static void setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 nelems, int op_type) + JsonbValue *newval, uint32 nelems, int op_type) { JsonbValue v; int idx, @@ -5017,7 +5038,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, (op_type & JB_PATH_CREATE_OR_INSERT)) { Assert(newval != NULL); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); done = true; } @@ -5033,7 +5054,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, r = JsonbIteratorNext(it, &v, true); /* skip */ if (op_type & (JB_PATH_INSERT_BEFORE | JB_PATH_CREATE)) - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); /* * We should keep current value only in case of @@ -5044,7 +5065,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, (void) pushJsonbValue(st, r, &v); if (op_type & (JB_PATH_INSERT_AFTER | JB_PATH_REPLACE)) - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); done = true; } @@ -5078,12 +5099,138 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, if ((op_type & JB_PATH_CREATE_OR_INSERT) && !done && level == path_len - 1 && i == nelems - 1) { - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); } } } } +/* + * Perform an actual data extraction or modification for the jsonb + * subscripting. As a result the extracted Datum or the modified containers + * value will be returned. + */ +Datum +jsonb_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) +{ + return jsonb_get_element(DatumGetJsonbP(containerSource), + sbstate->upperindex, + sbstate->numupper, + &sbstate->resnull, + false); +} + + + +/* + * Perform an actual data extraction or modification for the jsonb + * subscripting. As a result the extracted Datum or the modified containers + * value will be returned. + */ +Datum +jsonb_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) +{ + /* + * the original jsonb must be non-NULL, else we punt and return the + * original array. + */ + if (sbstate->resnull) + return containerSource; + + return jsonb_set_element(containerSource, + sbstate->upperindex, + sbstate->numupper, + sbstate->replacevalue, + sbstate->refelemtype, + sbstate->replacenull); +} + +/* + * Perform preparation for the jsonb subscripting. Since there are not any + * particular restrictions for this kind of subscripting, we will verify that + * it is not a slice operation. This function produces an expression that + * represents the result of extracting a single container element or the new + * container value with the source data inserted into the right part of the + * container. If you have read until this point, and will submit a meaningful + * review of this patch series, I'll owe you a beer at the next PGConfEU. + */ + +/* + * Handle jsonb-type subscripting logic. + */ +Datum +jsonb_subscript_handler(PG_FUNCTION_ARGS) +{ + SubscriptRoutines *sbsroutines = (SubscriptRoutines *) + palloc(sizeof(SubscriptRoutines)); + + sbsroutines->prepare = jsonb_subscript_prepare; + sbsroutines->validate = jsonb_subscript_validate; + sbsroutines->fetch = jsonb_subscript_fetch; + sbsroutines->assign = jsonb_subscript_assign; + + PG_RETURN_POINTER(sbsroutines); +} + +SubscriptingRef * +jsonb_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref) +{ + if (isAssignment) + { + sbsref->refelemtype = exprType((Node *) sbsref->refassgnexpr); + sbsref->refassgntype = exprType((Node *) sbsref->refassgnexpr); + } + else + sbsref->refelemtype = JSONBOID; + + return sbsref; +} + +SubscriptingRef * +jsonb_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate) +{ + List *upperIndexpr = NIL; + ListCell *l; + + if (sbsref->reflowerindexpr != NIL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript does not support slices"), + parser_errposition(pstate, exprLocation( + ((Node *) linitial(sbsref->reflowerindexpr)))))); + + foreach(l, sbsref->refupperindexpr) + { + Node *subexpr = (Node *) lfirst(l); + + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript does not support slices"), + parser_errposition(pstate, exprLocation( + ((Node *) linitial(sbsref->refupperindexpr)))))); + + subexpr = coerce_to_target_type(pstate, + subexpr, exprType(subexpr), + TEXTOID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript must have text type"), + parser_errposition(pstate, exprLocation(subexpr)))); + + upperIndexpr = lappend(upperIndexpr, subexpr); + } + + sbsref->refupperindexpr = upperIndexpr; + + return sbsref; +} + /* * Parse information about what elements of a jsonb document we want to iterate * in functions iterate_json(b)_values. This information is presented in jsonb diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 6de2dd29f1..36637afefb 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10707,6 +10707,14 @@ proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version}', prosrc => 'pg_control_init' }, +# type subscripting support +{ oid => '4191', + descr => 'Jsonb subscripting logic', + proname => 'jsonb_subscript_handler', + prorettype => 'internal', + proargtypes => 'internal', + prosrc => 'jsonb_subscript_handler' }, + { oid => '4192', descr => 'Array subscripting logic', proname => 'array_subscript_handler', diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index f868a6d52e..64c0c6d519 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -447,7 +447,8 @@ { oid => '3802', array_type_oid => '3807', descr => 'Binary JSON', typname => 'jsonb', typlen => '-1', typbyval => 'f', typcategory => 'U', typinput => 'jsonb_in', typoutput => 'jsonb_out', typreceive => 'jsonb_recv', - typsend => 'jsonb_send', typalign => 'i', typstorage => 'x' }, + typsend => 'jsonb_send', typalign => 'i', typstorage => 'x', + typsubshandler => 'jsonb_subscript_handler' }, { oid => '4072', array_type_oid => '4073', descr => 'JSON path', typname => 'jsonpath', typlen => '-1', typbyval => 'f', typcategory => 'U', typinput => 'jsonpath_in', typoutput => 'jsonpath_out', diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h index 5860011693..6e3b75d56a 100644 --- a/src/include/utils/jsonb.h +++ b/src/include/utils/jsonb.h @@ -392,6 +392,7 @@ extern JsonbValue *pushJsonbValue(JsonbParseState **pstate, extern JsonbIterator *JsonbIteratorInit(JsonbContainer *container); extern JsonbIteratorToken JsonbIteratorNext(JsonbIterator **it, JsonbValue *val, bool skipNested); +extern JsonbValue *JsonbToJsonbValue(Jsonb *jsonb); extern Jsonb *JsonbValueToJsonb(JsonbValue *val); extern bool JsonbDeepContains(JsonbIterator **val, JsonbIterator **mContained); @@ -407,5 +408,6 @@ extern char *JsonbToCStringIndent(StringInfo out, JsonbContainer *in, extern bool JsonbExtractScalar(JsonbContainer *jbc, JsonbValue *res); extern const char *JsonbTypeName(JsonbValue *jb); +extern JsonbValue *to_jsonb_worker(Datum source, Oid source_type, bool is_null); #endif /* __JSONB_H__ */ diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out index a70cd0b7c1..04a146a7d0 100644 --- a/src/test/regress/expected/jsonb.out +++ b/src/test/regress/expected/jsonb.out @@ -4567,7 +4567,7 @@ select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'use_ {"a": 1, "b": null} (1 row) -\pset null +\pset null '' -- jsonb_insert select jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"'); jsonb_insert @@ -4697,6 +4697,237 @@ HINT: Try using the function jsonb_set to replace key value. select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"', true); ERROR: cannot replace existing key HINT: Try using the function jsonb_set to replace key value. +-- jsonb subscript +select ('123'::jsonb)['a']; + jsonb +------- + +(1 row) + +select ('123'::jsonb)[0]; + jsonb +------- + +(1 row) + +select ('123'::jsonb)[NULL]; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)['a']; + jsonb +------- + 1 +(1 row) + +select ('{"a": 1}'::jsonb)[0]; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)['not_exist']; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)[NULL]; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)['a']; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[0]; + jsonb +------- + 1 +(1 row) + +select ('[1, "2", null]'::jsonb)['1']; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1.0]; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[2]; + jsonb +------- + null +(1 row) + +select ('[1, "2", null]'::jsonb)[3]; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[-2]; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1]['a']; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[1][0]; + jsonb +------- + +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['b']; + jsonb +------- + "c" +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']; + jsonb +----------- + [1, 2, 3] +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d'][1]; + jsonb +------- + 2 +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']['a']; + jsonb +------- + +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']; + jsonb +--------------- + {"a2": "aaa"} +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']; + jsonb +------- + "aaa" +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']['a3']; + jsonb +------- + +(1 row) + +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1']; + jsonb +----------------------- + ["aaa", "bbb", "ccc"] +(1 row) + +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1'][2]; + jsonb +------- + "ccc" +(1 row) + +create TEMP TABLE test_jsonb_subscript ( + id int, + test_json jsonb +); +insert into test_jsonb_subscript values +(1, '{}'), -- empty jsonb +(2, '{"key": "value"}'); -- jsonb with data +-- update empty jsonb +update test_jsonb_subscript set test_json['a'] = 1 where id = 1; +select * from test_jsonb_subscript; + id | test_json +----+------------------ + 2 | {"key": "value"} + 1 | {"a": 1} +(2 rows) + +-- update jsonb with some data +update test_jsonb_subscript set test_json['a'] = 1 where id = 2; +select * from test_jsonb_subscript; + id | test_json +----+-------------------------- + 1 | {"a": 1} + 2 | {"a": 1, "key": "value"} +(2 rows) + +-- replace jsonb +update test_jsonb_subscript set test_json['a'] = 'test'; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------- + 1 | {"a": "test"} + 2 | {"a": "test", "key": "value"} +(2 rows) + +-- replace by object +update test_jsonb_subscript set test_json['a'] = '{"b": 1}'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------- + 1 | {"a": {"b": 1}} + 2 | {"a": {"b": 1}, "key": "value"} +(2 rows) + +-- replace by array +update test_jsonb_subscript set test_json['a'] = '[1, 2, 3]'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+---------------------------------- + 1 | {"a": [1, 2, 3]} + 2 | {"a": [1, 2, 3], "key": "value"} +(2 rows) + +-- use jsonb subscription in where clause +select * from test_jsonb_subscript where test_json['key'] = '"value"'; + id | test_json +----+---------------------------------- + 2 | {"a": [1, 2, 3], "key": "value"} +(1 row) + +select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; + id | test_json +----+----------- +(0 rows) + +select * from test_jsonb_subscript where test_json['key'] = '"wrong_value"'; + id | test_json +----+----------- +(0 rows) + +-- NULL +update test_jsonb_subscript set test_json[NULL] = 1; +ERROR: subscript in assignment must not be null +update test_jsonb_subscript set test_json['another_key'] = NULL; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------------------- + 1 | {"a": [1, 2, 3], "another_key": null} + 2 | {"a": [1, 2, 3], "key": "value", "another_key": null} +(2 rows) + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); to_tsvector diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql index 3e2b8f66df..12541e7e50 100644 --- a/src/test/regress/sql/jsonb.sql +++ b/src/test/regress/sql/jsonb.sql @@ -1172,7 +1172,7 @@ select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'retu select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'delete_key') as delete_key; select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'use_json_null') as use_json_null; -\pset null +\pset null '' -- jsonb_insert select jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"'); @@ -1203,6 +1203,72 @@ select jsonb_insert('{"a": {"b": "value"}}', '{a, c}', '"new_value"', true); select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"'); select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"', true); +-- jsonb subscript +select ('123'::jsonb)['a']; +select ('123'::jsonb)[0]; +select ('123'::jsonb)[NULL]; +select ('{"a": 1}'::jsonb)['a']; +select ('{"a": 1}'::jsonb)[0]; +select ('{"a": 1}'::jsonb)['not_exist']; +select ('{"a": 1}'::jsonb)[NULL]; +select ('[1, "2", null]'::jsonb)['a']; +select ('[1, "2", null]'::jsonb)[0]; +select ('[1, "2", null]'::jsonb)['1']; +select ('[1, "2", null]'::jsonb)[1.0]; +select ('[1, "2", null]'::jsonb)[2]; +select ('[1, "2", null]'::jsonb)[3]; +select ('[1, "2", null]'::jsonb)[-2]; +select ('[1, "2", null]'::jsonb)[1]['a']; +select ('[1, "2", null]'::jsonb)[1][0]; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['b']; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d'][1]; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']['a']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']['a3']; +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1']; +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1'][2]; + +create TEMP TABLE test_jsonb_subscript ( + id int, + test_json jsonb +); + +insert into test_jsonb_subscript values +(1, '{}'), -- empty jsonb +(2, '{"key": "value"}'); -- jsonb with data + +-- update empty jsonb +update test_jsonb_subscript set test_json['a'] = 1 where id = 1; +select * from test_jsonb_subscript; + +-- update jsonb with some data +update test_jsonb_subscript set test_json['a'] = 1 where id = 2; +select * from test_jsonb_subscript; + +-- replace jsonb +update test_jsonb_subscript set test_json['a'] = 'test'; +select * from test_jsonb_subscript; + +-- replace by object +update test_jsonb_subscript set test_json['a'] = '{"b": 1}'::jsonb; +select * from test_jsonb_subscript; + +-- replace by array +update test_jsonb_subscript set test_json['a'] = '[1, 2, 3]'::jsonb; +select * from test_jsonb_subscript; + +-- use jsonb subscription in where clause +select * from test_jsonb_subscript where test_json['key'] = '"value"'; +select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; +select * from test_jsonb_subscript where test_json['key'] = '"wrong_value"'; + +-- NULL +update test_jsonb_subscript set test_json[NULL] = 1; +update test_jsonb_subscript set test_json['another_key'] = NULL; +select * from test_jsonb_subscript; + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); -- 2.21.0 --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0004-Subscripting-documentation.patch" From 664d34e46ec97a183e73a0bb375bf4c8bce7d88f Mon Sep 17 00:00:00 2001 From: erthalion <9erthalion6@gmail.com> Date: Fri, 1 Feb 2019 11:47:37 +0100 Subject: [PATCH v30 4/6] Subscripting documentation Supporting documentation for generalized subscripting. It includes the description of a new field in pg_type, the new section for jsonb documentation about subscripting feature on this data type, and also the tutorial about how to write subscripting operator for a custom data type. Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin --- doc/src/sgml/catalogs.sgml | 8 ++ doc/src/sgml/extend.sgml | 6 + doc/src/sgml/filelist.sgml | 1 + doc/src/sgml/json.sgml | 39 ++++++ doc/src/sgml/ref/create_type.sgml | 33 ++++- doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++ src/tutorial/Makefile | 4 +- src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++ src/tutorial/subscripting.source | 71 +++++++++++ 9 files changed, 470 insertions(+), 4 deletions(-) create mode 100644 doc/src/sgml/xsubscripting.sgml create mode 100644 src/tutorial/subscripting.c create mode 100644 src/tutorial/subscripting.source diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 34bc0d0526..328a1da6fe 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -7911,6 +7911,14 @@ SCRAM-SHA-256$<iteration count>:&l + + typsubshandler + regproc + pg_proc.oid + Custom subscripting function with type-specific logic for parsing + and validation, or 0 if this type doesn't support subscripting. + + typdefaultbin pg_node_tree diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 9ec1af780b..057010157e 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -33,6 +33,11 @@ operators (starting in ) + + + subscripting procedure (starting in ) + + operator classes for indexes (starting in ) @@ -314,6 +319,7 @@ &xaggr; &xtypes; &xoper; + &xsubscripting; &xindex; diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index 3da2365ea9..650e21b7e1 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -69,6 +69,7 @@ + diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml index 1b6aaf0a55..189f03b41e 100644 --- a/doc/src/sgml/json.sgml +++ b/doc/src/sgml/json.sgml @@ -600,6 +600,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu + + <type>jsonb</type> Subscripting + + jsonb data type supports array-style subscripting expressions + to extract or update particular element. It's possible to use multiple + subscripting expressions to extract nested values. In this case a chain of + subscripting expressions follows the same rules as the + path argument in jsonb_set function, + e.g. in case of arrays it is a 0-based operation or that negative integers + that appear in path count from the end of JSON arrays. + The result of subscripting expressions is always jsonb data type. An + example of subscripting syntax: + +-- Extract value by key +SELECT ('{"a": 1}'::jsonb)['a']; + +-- Extract nested value by key path +SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c']; + +-- Extract element by index +SELECT ('[1, "2", null]'::jsonb)[1]; + +-- Update value by key +UPDATE table_name SET jsonb_field['key'] = 1; + +-- Select records using where clause with subscripting. Since the result of +-- subscripting is jsonb and we basically want to compare two jsonb objects, we +-- need to put the value in double quotes to be able to convert it to jsonb. +SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"'; + + + There is no special indexing support for such kind of expressions, but you + always can create a functional index that includes it + +CREATE INDEX idx ON table_name ((jsonb_field['key'])); + + + + Transforms diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml index 175315f3d7..bd622f5ef3 100644 --- a/doc/src/sgml/ref/create_type.sgml +++ b/doc/src/sgml/ref/create_type.sgml @@ -54,6 +54,7 @@ CREATE TYPE name ( [ , ELEMENT = element ] [ , DELIMITER = delimiter ] [ , COLLATABLE = collatable ] + [ , SUBSCRIPTING_HANDLER = subscripting_handler_function ] ) CREATE TYPE name @@ -196,8 +197,9 @@ CREATE TYPE name receive_function, send_function, type_modifier_input_function, - type_modifier_output_function and - analyze_function + type_modifier_output_function, + analyze_function, + subscripting_handler_function are optional. Generally these functions have to be coded in C or another low-level language. @@ -454,6 +456,22 @@ CREATE TYPE name make use of the collation information; this does not happen automatically merely by marking the type collatable. + + + The optional + subscripting_handler_function + contains type-specific logic for subscripting of the data type. + By default, there is no such function provided, which means that the data + type doesn't support subscripting. The subscripting function must be + declared to take a single argument of type internal, and return + a internal result. There are two examples of implementation for + subscripting functions in case of array + (array_subscripting_handler) + and jsonb + (jsonb_subscripting_handler) + types in src/backend/utils/adt/arrayfuncs.c and + src/backend/utils/adt/jsonfuncs.c corresponding. + @@ -769,6 +787,17 @@ CREATE TYPE name + + + subscripting_handler_function + + + The name of a function that returns list of type-specific callback functions to + support subscripting logic for the data type. + + + + diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml new file mode 100644 index 0000000000..d701631223 --- /dev/null +++ b/doc/src/sgml/xsubscripting.sgml @@ -0,0 +1,111 @@ + + + + User-defined subscripting procedure + + + custom subscripting + + + When you define a new base type, you can also specify a custom procedures to + handle subscripting expressions. They must contain logic for verification and + evaluation of this expression, i.e. fetching or updating some data in this + data type. For instance: + +prepare = custom_subscript_prepare; + sbsroutines->validate = custom_subscript_validate; + sbsroutines->fetch = custom_subscript_fetch; + sbsroutines->assign = custom_subscript_assign; + + PG_RETURN_POINTER(sbsroutines); +} + +SubscriptingRef * +custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref) +{ + sbsref->refelemtype = someType; + sbsref->refassgntype = someType; + + return sbsref; +} + +SubscriptingRef * +custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate) +{ + // some validation and coercion logic + + return sbsref; +} + +Datum +custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) +{ + // Some assignment logic + + return newContainer; +} + +Datum +custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) +{ + // Some fetch logic based on sbsdata +}]]> + + + + Then you can define a subscripting procedures and a custom data type: + + +CREATE FUNCTION custom_subscripting_handler(internal) + RETURNS internal + AS 'filename' + LANGUAGE C IMMUTABLE STRICT; + +CREATE TYPE custom ( + internallength = 4, + input = custom_in, + output = custom_out, + subscripting_handler = custom_subscripting_handler, +); + + + + and use it as usual: + + +CREATE TABLE test_subscripting ( + data custom +); + +INSERT INTO test_subscripting VALUES ('(1, 2)'); + +SELECT data[0] from test_subscripting; + +UPDATE test_subscripting SET data[1] = 3; + + + + + The examples of custom subscripting implementation can be found in + subscripting.sql and subscripting.c + in the src/tutorial directory of the source distribution. + See the README file in that directory for instructions + about running the examples. + + + diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile index 16dc390f71..0ead60c2d4 100644 --- a/src/tutorial/Makefile +++ b/src/tutorial/Makefile @@ -13,8 +13,8 @@ # #------------------------------------------------------------------------- -MODULES = complex funcs -DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql +MODULES = complex funcs subscripting +DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql ifdef NO_PGXS subdir = src/tutorial diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c new file mode 100644 index 0000000000..1eb8c45652 --- /dev/null +++ b/src/tutorial/subscripting.c @@ -0,0 +1,201 @@ +/* + * src/tutorial/subscripting.c + * + ****************************************************************************** + This file contains routines that can be bound to a Postgres backend and + called by the backend in the process of processing queries. The calling + format for these routines is dictated by Postgres architecture. +******************************************************************************/ + +#include "postgres.h" + +#include "catalog/pg_type.h" +#include "executor/executor.h" +#include "executor/execExpr.h" +#include "nodes/nodeFuncs.h" +#include "parser/parse_coerce.h" +#include "utils/builtins.h" +#include "utils/fmgrprotos.h" + +PG_MODULE_MAGIC; + +typedef struct Custom +{ + int first; + int second; +} Custom; + +SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref); +SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate); +Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate); +Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate); + +PG_FUNCTION_INFO_V1(custom_in); +PG_FUNCTION_INFO_V1(custom_out); +PG_FUNCTION_INFO_V1(custom_subscripting_handler); + +/***************************************************************************** + * Input/Output functions + *****************************************************************************/ + +Datum +custom_in(PG_FUNCTION_ARGS) +{ + char *str = PG_GETARG_CSTRING(0); + int firstValue, + secondValue; + Custom *result; + + if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid input syntax for complex: \"%s\"", + str))); + + + result = (Custom *) palloc(sizeof(Custom)); + result->first = firstValue; + result->second = secondValue; + PG_RETURN_POINTER(result); +} + +Datum +custom_out(PG_FUNCTION_ARGS) +{ + Custom *custom = (Custom *) PG_GETARG_POINTER(0); + char *result; + + result = psprintf("(%d, %d)", custom->first, custom->second); + PG_RETURN_CSTRING(result); +} + +/***************************************************************************** + * Custom subscripting logic functions + *****************************************************************************/ + +Datum +custom_subscripting_handler(PG_FUNCTION_ARGS) +{ + SubscriptRoutines *sbsroutines = (SubscriptRoutines *) + palloc(sizeof(SubscriptRoutines)); + + sbsroutines->prepare = custom_subscript_prepare; + sbsroutines->validate = custom_subscript_validate; + sbsroutines->fetch = custom_subscript_fetch; + sbsroutines->assign = custom_subscript_assign; + + PG_RETURN_POINTER(sbsroutines); +} + +SubscriptingRef * +custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref) +{ + sbsref->refelemtype = INT4OID; + sbsref->refassgntype = INT4OID; + return sbsref; +} + +SubscriptingRef * +custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, + ParseState *pstate) +{ + List *upperIndexpr = NIL; + ListCell *l; + + if (sbsref->reflowerindexpr != NIL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("custom subscript does not support slices"), + parser_errposition(pstate, exprLocation( + ((Node *)lfirst(sbsref->reflowerindexpr->head)))))); + + foreach(l, sbsref->refupperindexpr) + { + Node *subexpr = (Node *) lfirst(l); + + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("custom subscript does not support slices"), + parser_errposition(pstate, exprLocation( + ((Node *) lfirst(sbsref->refupperindexpr->head)))))); + + subexpr = coerce_to_target_type(pstate, + subexpr, exprType(subexpr), + INT4OID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (subexpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("custom subscript must have integer type"), + parser_errposition(pstate, exprLocation(subexpr)))); + + upperIndexpr = lappend(upperIndexpr, subexpr); + + if (isAssignment) + { + Node *assignExpr = (Node *) sbsref->refassgnexpr; + Node *new_from; + + new_from = coerce_to_target_type(pstate, + assignExpr, exprType(assignExpr), + INT4OID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + if (new_from == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("custom assignment requires int type"), + errhint("You will need to rewrite or cast the expression."), + parser_errposition(pstate, exprLocation(assignExpr)))); + sbsref->refassgnexpr = (Expr *)new_from; + } + } + + sbsref->refupperindexpr = upperIndexpr; + + return sbsref; +} + +Datum +custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) +{ + Custom *container= (Custom *) containerSource; + int index; + + if (sbstate->numupper != 1) + ereport(ERROR, (errmsg("custom does not support nested subscripting"))); + + index = DatumGetInt32(sbstate->upperindex[0]); + + if (index == 1) + return (Datum) container->first; + else + return (Datum) container->second; +} + +Datum +custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) +{ + int index; + Custom *container = (Custom *) containerSource; + + if (sbstate->resnull) + return containerSource; + + if (sbstate->numupper != 1) + ereport(ERROR, (errmsg("custom does not support nested subscripting"))); + + index = DatumGetInt32(sbstate->upperindex[0]); + + if (index == 1) + container->first = DatumGetInt32(sbstate->replacevalue); + else + container->second = DatumGetInt32(sbstate->replacevalue); + + return (Datum) container; +} diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source new file mode 100644 index 0000000000..837cf30612 --- /dev/null +++ b/src/tutorial/subscripting.source @@ -0,0 +1,71 @@ +--------------------------------------------------------------------------- +-- +-- subscripting.sql- +-- This file shows how to create a new subscripting procedure for +-- user-defined type. +-- +-- +-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group +-- Portions Copyright (c) 1994, Regents of the University of California +-- +-- src/tutorial/subscripting.source +-- +--------------------------------------------------------------------------- + +----------------------------- +-- Creating a new type: +-- We are going to create a new type called 'complex' which represents +-- complex numbers. +-- A user-defined type must have an input and an output function, and +-- optionally can have binary input and output functions. All of these +-- are usually user-defined C functions. +----------------------------- + +-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX +-- (we do not want to assume this is in the dynamic loader search path). +-- Look at $PWD/complex.c for the source. Note that we declare all of +-- them as STRICT, so we do not need to cope with NULL inputs in the +-- C code. We also mark them IMMUTABLE, since they always return the +-- same outputs given the same inputs. + +-- the input function 'complex_in' takes a null-terminated string (the +-- textual representation of the type) and turns it into the internal +-- (in memory) representation. You will get a message telling you 'complex' +-- does not exist yet but that's okay. + +CREATE FUNCTION custom_in(cstring) + RETURNS custom + AS '_OBJWD_/subscripting' + LANGUAGE C IMMUTABLE STRICT; + +-- the output function 'complex_out' takes the internal representation and +-- converts it into the textual representation. + +CREATE FUNCTION custom_out(custom) + RETURNS cstring + AS '_OBJWD_/subscripting' + LANGUAGE C IMMUTABLE STRICT; + +CREATE FUNCTION custom_subscripting_handler(internal) + RETURNS internal + AS '_OBJWD_/subscripting' + LANGUAGE C IMMUTABLE STRICT; + +CREATE TYPE custom ( + internallength = 8, + input = custom_in, + output = custom_out, + subscripting_handler = custom_subscripting_handler +); + +-- we can use it in a table + +CREATE TABLE test_subscripting ( + data custom +); + +INSERT INTO test_subscripting VALUES ('(1, 2)'); + +SELECT data[0] from test_subscripting; + +UPDATE test_subscripting SET data[1] = 3; -- 2.21.0 --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0005-Polymorphic-subscripting.patch" From 8d69709a04d32bbed8505efff4b2bb40c9334b84 Mon Sep 17 00:00:00 2001 From: Dmitrii Dolgov <9erthalion6@gmail.com> Date: Thu, 19 Dec 2019 14:08:59 +0100 Subject: [PATCH v30 5/6] Polymorphic subscripting To improve performance and compliance with the SQL standard (in regards how float indexes are considered), interpret each subscript expression variant depending on the result of previous subscripting step. There are two variants of jsonb subscript expressions - the first is casted to text and the second is casted to int4. Executor at each subscripting step selects which variant to execute by calling callback jsonb_subscript_selectexpr(). To manage the subscripting state, another callback jsonb_subscript_step() was introduced along with the new field SubscriptingRefState.privatedata. Author: Nikita Glukhov --- src/backend/executor/execExpr.c | 226 ++++++-- src/backend/executor/execExprInterp.c | 87 ++- src/backend/utils/adt/arrayfuncs.c | 4 +- src/backend/utils/adt/jsonb_util.c | 6 +- src/backend/utils/adt/jsonfuncs.c | 785 ++++++++++++++++++++------ src/backend/utils/adt/jsonpath_exec.c | 2 +- src/include/executor/execExpr.h | 26 +- src/include/nodes/subscripting.h | 14 +- src/include/utils/jsonb.h | 2 +- src/test/regress/expected/jsonb.out | 233 +++++++- src/test/regress/sql/jsonb.sql | 80 +++ 11 files changed, 1211 insertions(+), 254 deletions(-) diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c index a756f4ba8d..96b2edf2e1 100644 --- a/src/backend/executor/execExpr.c +++ b/src/backend/executor/execExpr.c @@ -2534,6 +2534,144 @@ ExecInitWholeRowVar(ExprEvalStep *scratch, Var *variable, ExprState *state) } } +static void +ExecInitSubscriptExpr(ExprEvalStep *scratch, SubscriptingRefState *sbsrefstate, + ExprState *state, Node *expr, int off, bool isupper, + List **adjust_jumps) +{ + /* Each subscript is evaluated into subscriptvalue/subscriptnull */ + ExecInitExprRec((Expr *) expr, state, + &sbsrefstate->subscriptvalue, + &sbsrefstate->subscriptnull); + + /* ... and then SBSREF_SUBSCRIPT saves it into step's workspace */ + scratch->opcode = EEOP_SBSREF_SUBSCRIPT; + scratch->d.sbsref_subscript.state = sbsrefstate; + scratch->d.sbsref_subscript.typid = exprType(expr); + scratch->d.sbsref_subscript.off = off; + scratch->d.sbsref_subscript.isupper = isupper; + scratch->d.sbsref_subscript.jumpdone = -1; /* adjust later */ + ExprEvalPushStep(state, scratch); + + *adjust_jumps = lappend_int(*adjust_jumps, state->steps_len - 1); +} + +/* Init subscript expressions. */ +static void +ExecInitSubscript(ExprEvalStep *scratch, SubscriptingRefState *sbsrefstate, + ExprState *state, Node *expr, int i, bool isupper, + List **adjust_jumps, bool *isprovided, Oid *exprtype) +{ + List *exprs = NULL; + int nexprs = 0; + int select_step; + + /* When slicing, individual subscript bounds can be omitted */ + *isprovided = expr != NULL; + if (!*isprovided) + return; + + /* + * Node can be a list of expression variants. The first variant is + * an unmodified expression, other variants can be NULL, so we need + * to check if there are any non-NULL and emit SELECTEXPR if any. + */ + if (IsA(expr, List)) + { + exprs = (List *) expr; + expr = linitial(exprs); + + if (list_length(exprs) > 1) + { + ListCell *lc = list_head(exprs); + + while ((lc = lnext(exprs, lc))) + { + if (lfirst(lc)) + { + nexprs = list_length(exprs) - 1; + break; + } + } + } + } + + *exprtype = exprType(expr); + + /* Emit SELECTEXPR step if there are expression variants */ + if (nexprs) + { + scratch->opcode = EEOP_SBSREF_SELECTEXPR; + scratch->d.sbsref_selectexpr.state = sbsrefstate; + scratch->d.sbsref_selectexpr.off = i; + scratch->d.sbsref_selectexpr.isupper = isupper; + scratch->d.sbsref_selectexpr.nexprs = nexprs; + scratch->d.sbsref_selectexpr.exprtypes = palloc(sizeof(Oid) * nexprs); + scratch->d.sbsref_selectexpr.jumpdones = palloc(sizeof(int) * nexprs); + ExprEvalPushStep(state, scratch); + select_step = state->steps_len - 1; + } + + /* Emit main expression */ + ExecInitSubscriptExpr(scratch, sbsrefstate, state, expr, i, isupper, + adjust_jumps); + + /* Emit additional expression variants, if any */ + if (nexprs) + { + ListCell *lc = list_head(exprs); + List *adjust_subexpr_jumps = NIL; + int j = 0; + + /* Skip first expression which is already emitted */ + while ((lc = lnext(exprs, lc))) + { + int jumpdone; + Oid exprtype; + ExprEvalStep *step; + + expr = lfirst(lc); + + if (expr) + { + /* Emit JUMP to the end for previous expression */ + scratch->opcode = EEOP_JUMP; + scratch->d.jump.jumpdone = -1; /* adjust later */ + ExprEvalPushStep(state, scratch); + + adjust_subexpr_jumps = lappend_int(adjust_subexpr_jumps, + state->steps_len - 1); + + exprtype = exprType((Node *) expr); + jumpdone = state->steps_len; + + ExecInitSubscriptExpr(scratch, sbsrefstate, state, expr, + i, isupper, adjust_jumps); + } + else + { + exprtype = InvalidOid; + jumpdone = -1; + } + + step = &state->steps[select_step]; + step->d.sbsref_selectexpr.exprtypes[j] = exprtype; + step->d.sbsref_selectexpr.jumpdones[j] = jumpdone; + + j++; + } + + /* Adjust JUMPs for expression variants */ + foreach(lc, adjust_subexpr_jumps) + { + ExprEvalStep *step = &state->steps[lfirst_int(lc)]; + + Assert(step->opcode == EEOP_JUMP); + step->d.jump.jumpdone = state->steps_len; + } + } +} + /* * Prepare evaluation of a SubscriptingRef expression. */ @@ -2541,12 +2679,14 @@ static void ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, ExprState *state, Datum *resv, bool *resnull) { - bool isAssignment = (sbsref->refassgnexpr != NULL); SubscriptingRefState *sbsrefstate = palloc0(sizeof(SubscriptingRefState)); - List *adjust_jumps = NIL; - ListCell *lc; - int i; - RegProcedure typsubshandler = get_typsubsprocs(sbsref->refcontainertype); + List *adjust_jumps = NIL; + ListCell *lc; + ListCell *ulc; + ListCell *llc; + int i; + RegProcedure typsubshandler = get_typsubsprocs(sbsref->refcontainertype); + bool isAssignment = (sbsref->refassgnexpr != NULL); /* Fill constant fields of SubscriptingRefState */ sbsrefstate->isassignment = isAssignment; @@ -2576,71 +2716,43 @@ ExecInitSubscriptingRef(ExprEvalStep *scratch, SubscriptingRef *sbsref, state->steps_len - 1); } - /* Evaluate upper subscripts */ - i = 0; - foreach(lc, sbsref->refupperindexpr) + /* Emit INIT step if needed. */ + if (sbsrefstate->sbsroutines->init) { - Expr *e = (Expr *) lfirst(lc); - - /* When slicing, individual subscript bounds can be omitted */ - if (!e) - { - sbsrefstate->upperprovided[i] = false; - i++; - continue; - } - - sbsrefstate->upperprovided[i] = true; - - /* Each subscript is evaluated into subscriptvalue/subscriptnull */ - ExecInitExprRec(e, state, - &sbsrefstate->subscriptvalue, &sbsrefstate->subscriptnull); - - /* ... and then SBSREF_SUBSCRIPT saves it into step's workspace */ - scratch->opcode = EEOP_SBSREF_SUBSCRIPT; - scratch->d.sbsref_subscript.state = sbsrefstate; - scratch->d.sbsref_subscript.off = i; - scratch->d.sbsref_subscript.isupper = true; - scratch->d.sbsref_subscript.jumpdone = -1; /* adjust later */ + scratch->opcode = EEOP_SBSREF_INIT; + scratch->d.sbsref.state = sbsrefstate; ExprEvalPushStep(state, scratch); - adjust_jumps = lappend_int(adjust_jumps, - state->steps_len - 1); - i++; } - sbsrefstate->numupper = i; - /* Evaluate lower subscripts similarly */ + /* Evaluate upper and lower subscripts */ i = 0; - foreach(lc, sbsref->reflowerindexpr) + llc = list_head(sbsref->reflowerindexpr); + + sbsrefstate->numlower = 0; + + foreach(ulc, sbsref->refupperindexpr) { - Expr *e = (Expr *) lfirst(lc); + ExecInitSubscript(scratch, sbsrefstate, state, lfirst(ulc), i, + true, &adjust_jumps, + &sbsrefstate->upperprovided[i], + &sbsrefstate->uppertypid[i]); - /* When slicing, individual subscript bounds can be omitted */ - if (!e) + if (llc) { - sbsrefstate->lowerprovided[i] = false; - i++; - continue; - } + ExecInitSubscript(scratch, sbsrefstate, state, lfirst(llc), + i, false, &adjust_jumps, + &sbsrefstate->lowerprovided[i], + &sbsrefstate->lowertypid[i]); - sbsrefstate->lowerprovided[i] = true; + llc = lnext(sbsref->reflowerindexpr, llc); - /* Each subscript is evaluated into subscriptvalue/subscriptnull */ - ExecInitExprRec(e, state, - &sbsrefstate->subscriptvalue, &sbsrefstate->subscriptnull); + sbsrefstate->numlower++; + } - /* ... and then SBSREF_SUBSCRIPT saves it into step's workspace */ - scratch->opcode = EEOP_SBSREF_SUBSCRIPT; - scratch->d.sbsref_subscript.state = sbsrefstate; - scratch->d.sbsref_subscript.off = i; - scratch->d.sbsref_subscript.isupper = false; - scratch->d.sbsref_subscript.jumpdone = -1; /* adjust later */ - ExprEvalPushStep(state, scratch); - adjust_jumps = lappend_int(adjust_jumps, - state->steps_len - 1); i++; } - sbsrefstate->numlower = i; + + sbsrefstate->numupper = i; /* Should be impossible if parser is sane, but check anyway: */ if (sbsrefstate->numlower != 0 && diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c index 0a103bb403..841a15e4a8 100644 --- a/src/backend/executor/execExprInterp.c +++ b/src/backend/executor/execExprInterp.c @@ -416,6 +416,8 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull) &&CASE_EEOP_FIELDSELECT, &&CASE_EEOP_FIELDSTORE_DEFORM, &&CASE_EEOP_FIELDSTORE_FORM, + &&CASE_EEOP_SBSREF_INIT, + &&CASE_EEOP_SBSREF_SELECTEXPR, &&CASE_EEOP_SBSREF_SUBSCRIPT, &&CASE_EEOP_SBSREF_OLD, &&CASE_EEOP_SBSREF_ASSIGN, @@ -1395,6 +1397,29 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull) EEO_NEXT(); } + EEO_CASE(EEOP_SBSREF_INIT) + { + /* too complex for an inline implementation */ + ExecEvalSubscriptingRefInit(state, op); + + EEO_NEXT(); + } + + EEO_CASE(EEOP_SBSREF_SELECTEXPR) + { + /* too complex for an inline implementation */ + int selectedExpr = ExecEvalSubscriptingRefSelect(state, op); + + /* + * Jump to selected expression variant or simply continue + * to the first (0th) expression + */ + if (selectedExpr > 0) + EEO_JUMP(op->d.sbsref_selectexpr.jumpdones[selectedExpr - 1]); + else + EEO_NEXT(); + } + EEO_CASE(EEOP_SBSREF_SUBSCRIPT) { /* Process an array subscript */ @@ -3119,6 +3144,46 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext *op->resnull = false; } +/* + * Initialize subscripting state. + */ +void +ExecEvalSubscriptingRefInit(ExprState *state, ExprEvalStep *op) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref_selectexpr.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; + + /* init private subsripting state */ + sbsroutines->init(sbsrefstate, *op->resvalue, *op->resnull); +} + +/* + * Select expression variant for subscript evaluation + */ +int +ExecEvalSubscriptingRefSelect(ExprState *state, ExprEvalStep *op) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref_selectexpr.state; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; + Oid *typids = op->d.sbsref_selectexpr.isupper ? + sbsrefstate->uppertypid : sbsrefstate->lowertypid; + int off = op->d.sbsref_selectexpr.off; + Oid *exprtypes = op->d.sbsref_selectexpr.exprtypes; + Oid typid = typids[off]; + int selected; + + selected = sbsroutines->selectexpr(sbsrefstate, off, typid, exprtypes, + op->d.sbsref_selectexpr.nexprs); + + if (selected) + { + Assert(OidIsValid(exprtypes[selected])); + typids[off] = exprtypes[selected - 1]; + } + + return selected; +} + /* * Process a subscript in a SubscriptingRef expression. * @@ -3134,8 +3199,10 @@ bool ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) { SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; - Datum *indexes; - int off; + SubscriptRoutines *sbsroutines = sbsrefstate->sbsroutines; + Datum *indexes; + int off; + bool isupper; /* If any index expr yields NULL, result is NULL or error */ if (sbsrefstate->subscriptnull) @@ -3148,15 +3215,24 @@ ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op) return false; } - /* Convert datum to int, save in appropriate place */ - if (op->d.sbsref_subscript.isupper) + off = op->d.sbsref_subscript.off; + isupper = op->d.sbsref_subscript.isupper; + + /* Save converted datum in appropriate place */ + if (isupper) indexes = sbsrefstate->upperindex; else indexes = sbsrefstate->lowerindex; - off = op->d.sbsref_subscript.off; indexes[off] = sbsrefstate->subscriptvalue; + if (sbsroutines->step && + !sbsroutines->step(sbsrefstate, off, isupper)) + { + *op->resnull = true; + return false; + } + return true; } @@ -3174,7 +3250,6 @@ ExecEvalSubscriptingRefFetch(ExprState *state, ExprEvalStep *op) /* Should not get here if source container (or any subscript) is null */ Assert(!(*op->resnull)); - *op->resvalue = sbsroutines->fetch(*op->resvalue, sbsrefstate); *op->resnull = sbsrefstate->resnull; } diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 01bc3e8ac5..7dcef967cd 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -6729,7 +6729,7 @@ Datum array_subscript_handler(PG_FUNCTION_ARGS) { SubscriptRoutines *sbsroutines = (SubscriptRoutines *) - palloc(sizeof(SubscriptRoutines)); + palloc0(sizeof(SubscriptRoutines)); sbsroutines->prepare = array_subscript_prepare; sbsroutines->validate = array_subscript_validate; @@ -6863,7 +6863,7 @@ array_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("array assignment requires type %s" " but expression is of type %s", - format_type_be(sbsref->refelemtype), + format_type_be(typeneeded), format_type_be(typesource)), errhint("You will need to rewrite or cast the expression."), parser_errposition(pstate, exprLocation(assignExpr)))); diff --git a/src/backend/utils/adt/jsonb_util.c b/src/backend/utils/adt/jsonb_util.c index adcf16acf1..78664ed451 100644 --- a/src/backend/utils/adt/jsonb_util.c +++ b/src/backend/utils/adt/jsonb_util.c @@ -469,9 +469,8 @@ getKeyJsonValueFromContainer(JsonbContainer *container, * Returns palloc()'d copy of the value, or NULL if it does not exist. */ JsonbValue * -getIthJsonbValueFromContainer(JsonbContainer *container, uint32 i) +getIthJsonbValueFromContainer(JsonbContainer *container, uint32 i, JsonbValue *result) { - JsonbValue *result; char *base_addr; uint32 nelements; @@ -484,7 +483,8 @@ getIthJsonbValueFromContainer(JsonbContainer *container, uint32 i) if (i >= nelements) return NULL; - result = palloc(sizeof(JsonbValue)); + if (!result) + result = palloc(sizeof(JsonbValue)); fillJsonbValue(container, i, base_addr, getJsonbOffset(container, i), diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c index 538740aec9..82d4a98e7e 100644 --- a/src/backend/utils/adt/jsonfuncs.c +++ b/src/backend/utils/adt/jsonfuncs.c @@ -335,6 +335,23 @@ typedef struct JsObject static int report_json_context(JsonLexContext *lex); +/* state for assignment of a single subscript */ +typedef struct JsonbSubscriptState +{ + bool exists; /* does this element exist? */ + bool is_array; /* is it array or object? */ + int array_size; /* size of array */ + int array_index; /* index in array (negative means prepending) */ +} JsonbSubscriptState; + +/* state for subscript assignment */ +typedef struct JsonbAssignState +{ + JsonbParseState *ps; /* jsonb building state */ + JsonbIterator *iter; /* source jsonb iterator */ + JsonbSubscriptState subscripts[MAX_SUBSCRIPT_DEPTH + 1]; /* per-subscript states */ +} JsonbAssignState; + /* semantic action functions for json_object_keys */ static void okeys_object_field_start(void *state, char *fname, bool isnull); static void okeys_array_start(void *state); @@ -464,8 +481,6 @@ static Datum populate_domain(DomainIOData *io, Oid typid, const char *colname, /* functions supporting jsonb_delete, jsonb_set and jsonb_concat */ static JsonbValue *IteratorConcat(JsonbIterator **it1, JsonbIterator **it2, JsonbParseState **state); -static Datum jsonb_set_element(Datum datum, Datum *path, int path_len, - Datum sourceData, Oid source_type, bool is_null); static Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text); static JsonbValue *setPath(JsonbIterator **it, Datum *path_elems, @@ -474,12 +489,13 @@ static JsonbValue *setPath(JsonbIterator **it, Datum *path_elems, int op_type); static void setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, - int level, - JsonbValue *newval, uint32 npairs, int op_type); + int level, JsonbValue *newval, int op_type); static void setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, JsonbValue *newval, uint32 nelems, int op_type); +static bool copyJsonbObject(JsonbParseState **st, JsonbIterator **it, + const JsonbValue *key); /* function supporting iterate_json_values */ static void iterate_values_scalar(void *state, char *token, JsonTokenType tokentype); @@ -932,6 +948,7 @@ jsonb_array_element(PG_FUNCTION_ARGS) Jsonb *jb = PG_GETARG_JSONB_P(0); int element = PG_GETARG_INT32(1); JsonbValue *v; + JsonbValue vbuf; if (!JB_ROOT_IS_ARRAY(jb)) PG_RETURN_NULL(); @@ -947,7 +964,7 @@ jsonb_array_element(PG_FUNCTION_ARGS) element += nelements; } - v = getIthJsonbValueFromContainer(&jb->root, element); + v = getIthJsonbValueFromContainer(&jb->root, element, &vbuf); if (v != NULL) PG_RETURN_JSONB_P(JsonbValueToJsonb(v)); @@ -975,6 +992,7 @@ jsonb_array_element_text(PG_FUNCTION_ARGS) Jsonb *jb = PG_GETARG_JSONB_P(0); int element = PG_GETARG_INT32(1); JsonbValue *v; + JsonbValue vbuf; if (!JB_ROOT_IS_ARRAY(jb)) PG_RETURN_NULL(); @@ -990,7 +1008,7 @@ jsonb_array_element_text(PG_FUNCTION_ARGS) element += nelements; } - v = getIthJsonbValueFromContainer(&jb->root, element); + v = getIthJsonbValueFromContainer(&jb->root, element, &vbuf); if (v != NULL && v->type != jbvNull) PG_RETURN_TEXT_P(JsonbValueAsText(v)); @@ -1010,6 +1028,26 @@ json_extract_path_text(PG_FUNCTION_ARGS) return get_path_all(fcinfo, true); } +static inline bool +jsonb_get_array_index_from_cstring(char *indexstr, long *index) +{ + char *endptr; + + errno = 0; + *index = strtol(indexstr, &endptr, 10); + if (endptr == indexstr || *endptr != '\0' || errno != 0 || + *index > INT_MAX || *index < INT_MIN) + return false; + + return true; +} + +static inline bool +jsonb_get_array_index_from_text(Datum indextext, long *index) +{ + return jsonb_get_array_index_from_cstring(TextDatumGetCString(indextext), index); +} + /* * common routine for extract_path functions */ @@ -1055,11 +1093,8 @@ get_path_all(FunctionCallInfo fcinfo, bool as_text) if (*tpath[i] != '\0') { long ind; - char *endptr; - errno = 0; - ind = strtol(tpath[i], &endptr, 10); - if (*endptr == '\0' && errno == 0 && ind <= INT_MAX && ind >= INT_MIN) + if (jsonb_get_array_index_from_cstring(tpath[i], &ind)) ipath[i] = (int) ind; else ipath[i] = INT_MIN; @@ -1503,13 +1538,362 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) PG_RETURN_DATUM(res); } +/* Initialize private jsonb subscripting state. */ +static void +jsonb_subscript_init(SubscriptingRefState *sbstate, Datum container, bool isnull) +{ + Jsonb *jb = isnull ? NULL : DatumGetJsonbP(container); + + if (sbstate->isassignment) + { + JsonbAssignState *astate = palloc0(sizeof(*astate)); + JsonbSubscriptState *subscript = &astate->subscripts[0]; + + astate->ps = NULL; + + if (jb) + { + JsonbValue jbv; + JsonbIteratorToken tok; + + astate->iter = JsonbIteratorInit(&jb->root); + + tok = JsonbIteratorNext(&astate->iter, &jbv, false); + + if (tok == WJB_BEGIN_ARRAY) + { + if (jbv.val.array.rawScalar) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("cannot assign to subscript of scalar jsonb"))); + + subscript->is_array = true; + subscript->array_size = jbv.val.array.nElems; + } + else + subscript->is_array = false; + + subscript->exists = true; + } + else + { + astate->iter = NULL; + subscript->exists = false; + } + + sbstate->privatedata = astate; + } + else + { + JsonbValue *jbv; + + /* Initialize a binary JsonbValue and use it as a private state */ + if (jb) + { + jbv = palloc(sizeof(*jbv)); + + jbv->type = jbvBinary; + jbv->val.binary.data = &jb->root; + jbv->val.binary.len = VARSIZE(jb) - VARHDRSZ; + } + else + jbv = NULL; + + sbstate->privatedata = jbv; + } +} + +/* + * Select subscript expression variant. + * + * There are two expression variants of jsonb subscripts: + * 0th - unmodified expression + * 1st - expression casted to int4 (optional, if type is numeric) + * + * If the current jsonb is an array then we select 1st variant, otherwise + * default 0th variant is selected. + */ +static int +jsonb_subscript_selectexpr(SubscriptingRefState *sbstate, int num, + Oid subscriptType, Oid *exprTypes, int nExprs) +{ + bool is_array = false; + + Assert(nExprs == 1); + Assert(!OidIsValid(exprTypes[0]) || exprTypes[0] == INT4OID); + + if (sbstate->isassignment) + { + JsonbAssignState *astate = sbstate->privatedata; + JsonbSubscriptState *subscript = &astate->subscripts[num]; + + if (!subscript->exists) + { + /* NULL can be only in assignments, select int4 variant if available. */ + Assert(sbstate->isassignment); + + subscript->is_array = OidIsValid(exprTypes[0]); + subscript->array_size = 0; + } + + is_array = subscript->is_array; + } + else + { + JsonbValue *jbv = sbstate->privatedata; + + Assert(jbv); + + if (jbv->type == jbvBinary) + { + JsonbContainer *jbc = jbv->val.binary.data; + + if (JsonContainerIsArray(jbc) && !JsonContainerIsScalar(jbc)) + is_array = true; + } + } + + if (is_array && OidIsValid(exprTypes[0])) + return 1; + + return 0; +} + +/* Get the integer index from a subscript datum */ +static int32 +jsonb_subscript_get_array_index(Datum value, Oid typid, int num, + int arraySize, bool isAssignment) +{ + long lindex; + + if (typid == INT4OID) + lindex = DatumGetInt32(value); + else if (typid != TEXTOID) + elog(ERROR, "invalid jsonb subscript type: %u", typid); + else if (!jsonb_get_array_index_from_text(value, &lindex)) + { + if (isAssignment) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("jsonb array subscript is not an integer: \"%s\"", + TextDatumGetCString(value)))); + return -1; + } + + if (lindex >= 0) + return lindex; + else /* handle negative subscript */ + return arraySize + lindex; +} + +/* Get the string key from a subscript datum */ +static char * +jsonb_subscript_get_object_key(Datum value, Oid typid, int *len) +{ + if (typid == TEXTOID) + { + *len = VARSIZE_ANY_EXHDR(value); + + return VARDATA_ANY(value); + } + else if (typid == INT4OID) + { + char *key = DatumGetCString(DirectFunctionCall1(int4out, value)); + + *len = strlen(key); + + return key; + } + else + { + elog(ERROR, "invalid jsonb subscript type: %u", typid); + return NULL; + } +} + +/* Apply single susbscript to jsonb container */ +static inline JsonbValue * +jsonb_subscript_apply(JsonbValue *jbv, Datum subscriptVal, Oid subscriptTypid, + int subscriptIdx) +{ + JsonbContainer *jbc; + + if (jbv->type != jbvBinary || + JsonContainerIsScalar(jbv->val.binary.data)) + return NULL; /* scalar, extraction yields a null */ + + jbc = jbv->val.binary.data; + + if (JsonContainerIsObject(jbc)) + { + int keylen; + char *keystr = jsonb_subscript_get_object_key(subscriptVal, + subscriptTypid, + &keylen); + + return getKeyJsonValueFromContainer(jbc, keystr, keylen, jbv); + } + else if (JsonContainerIsArray(jbc) && !JsonContainerIsScalar(jbc)) + { + int32 index = jsonb_subscript_get_array_index(subscriptVal, + subscriptTypid, + subscriptIdx, + JsonContainerSize(jbc), + false); + + if (index < 0) + return NULL; + + return getIthJsonbValueFromContainer(jbc, index, jbv); + } + else + { + /* scalar, extraction yields a null */ + return NULL; + } +} + +/* Perfrom one subscript assignment step */ +static void +jsonb_subscript_step_assignment(SubscriptingRefState *sbstate, Datum value, + Oid typid, int num, bool isupper) +{ + JsonbAssignState *astate = sbstate->privatedata; + JsonbSubscriptState *subscript = &astate->subscripts[num]; + JsonbIteratorToken tok; + JsonbValue jbv; + bool last = num >= sbstate->numupper - 1; + + if (!subscript->exists) + { + /* Select the type of newly created container. */ + if (typid == INT4OID) + { + subscript->is_array = true; + subscript->array_size = 0; + } + else if (typid == TEXTOID) + subscript->is_array = false; + else + elog(ERROR, "invalid jsonb subscript type: %u", typid); + } + + subscript[1].exists = false; + + if (subscript->is_array) + { + int32 i = 0; + int32 index = jsonb_subscript_get_array_index(value, typid, num, + subscript->array_size, + true); + + pushJsonbValue(&astate->ps, WJB_BEGIN_ARRAY, NULL); + + subscript->array_index = index; + + if (index >= 0 && subscript->exists) + { + /* Try to copy preceding elements */ + for (; i < index; i++) + { + tok = JsonbIteratorNext(&astate->iter, &jbv, true); + + if (tok != WJB_ELEM) + break; + + pushJsonbValue(&astate->ps, tok, &jbv); + } + + /* Try to read replaced element */ + if (i >= index && + JsonbIteratorNext(&astate->iter, &jbv, last) != WJB_END_ARRAY) + subscript[1].exists = true; + } + } + else + { + JsonbValue key; + + key.type = jbvString; + key.val.string.val = jsonb_subscript_get_object_key(value, typid, + &key.val.string.len); + + pushJsonbValue(&astate->ps, WJB_BEGIN_OBJECT, NULL); + + if (subscript->exists && + copyJsonbObject(&astate->ps, &astate->iter, &key)) + { + subscript[1].exists = true; /* key is found */ + tok = JsonbIteratorNext(&astate->iter, &jbv, last); + } + + pushJsonbValue(&astate->ps, WJB_KEY, &key); + } + + /* If the value does exits, process and validate its type. */ + if (subscript[1].exists) + { + if (jbv.type == jbvArray) + { + subscript[1].is_array = true; + subscript[1].array_size = jbv.val.array.nElems; + } + else + subscript[1].is_array = false; + + if (!last && IsAJsonbScalar(&jbv)) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("cannot assign to subscript of scalar jsonb"))); + } +} + +/* Perform one subscripting step */ +static bool +jsonb_subscript_step(SubscriptingRefState *sbstate, int num, bool isupper) +{ + Datum value; + Oid typid; + + if (isupper) + { + value = sbstate->upperindex[num]; + typid = sbstate->uppertypid[num]; + } + else + elog(ERROR, "jsonb subscript cannot be lower"); + + if (sbstate->isassignment) + { + jsonb_subscript_step_assignment(sbstate, value, typid, num, isupper); + + return true; /* always process next subscripts */ + } + else + { + /* + * Perform one subscripting step by applying subscript value to current + * jsonb container and saving the result into private state. + */ + JsonbValue *jbv = sbstate->privatedata; + + Assert(jbv); /* NULL can only be in assignments */ + + jbv = jsonb_subscript_apply(jbv, value, typid, num); + + sbstate->privatedata = jbv; + + /* Process next subscripts only if the result is not NULL */ + return jbv != NULL; + } +} + static Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) { - Jsonb *res; - JsonbContainer *container = &jb->root; + JsonbValue jbv; JsonbValue *jbvp = NULL; - JsonbValue tv; + JsonbContainer *container = &jb->root; int i; bool have_object = false, have_array = false; @@ -1517,16 +1901,16 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) *isnull = false; /* Identify whether we have object, array, or scalar at top-level */ - if (JB_ROOT_IS_OBJECT(jb)) + if (JsonContainerIsObject(container)) have_object = true; - else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb)) + else if (JsonContainerIsArray(container) && !JsonContainerIsScalar(container)) have_array = true; else { - Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb)); + Assert(JsonContainerIsArray(container) && JsonContainerIsScalar(container)); /* Extract the scalar value, if it is what we'll return */ if (npath <= 0) - jbvp = getIthJsonbValueFromContainer(container, 0); + jbvp = getIthJsonbValueFromContainer(container, 0, &jbv); } /* @@ -1559,19 +1943,14 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) jbvp = getKeyJsonValueFromContainer(container, VARDATA(path[i]), VARSIZE(path[i]) - VARHDRSZ, - NULL); + jbvp); } else if (have_array) { long lindex; uint32 index; - char *indextext = TextDatumGetCString(path[i]); - char *endptr; - errno = 0; - lindex = strtol(indextext, &endptr, 10); - if (endptr == indextext || *endptr != '\0' || errno != 0 || - lindex > INT_MAX || lindex < INT_MIN) + if (!jsonb_get_array_index_from_text(path[i], &lindex)) { *isnull = true; return PointerGetDatum(NULL); @@ -1601,7 +1980,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) index = nelements + lindex; } - jbvp = getIthJsonbValueFromContainer(container, index); + jbvp = getIthJsonbValueFromContainer(container, index, jbvp); } else { @@ -1652,32 +2031,6 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) } } -Datum -jsonb_set_element(Datum jsonbdatum, Datum *path, int path_len, - Datum sourceData, Oid source_type, bool is_null) -{ - Jsonb *jb = DatumGetJsonbP(jsonbdatum); - JsonbValue *newval, - *res; - JsonbParseState *state = NULL; - JsonbIterator *it; - bool *path_nulls = palloc0(path_len * sizeof(bool)); - - newval = to_jsonb_worker(sourceData, source_type, is_null); - - if (newval->type == jbvArray && newval->val.array.rawScalar) - *newval = newval->val.array.elems[0]; - - it = JsonbIteratorInit(&jb->root); - - res = setPath(&it, path, path_nulls, path_len, &state, 0, - newval, JB_PATH_CREATE); - - pfree(path_nulls); - - PG_RETURN_JSONB_P(JsonbValueToJsonb(res)); -} - /* * Return the text representation of the given JsonbValue. */ @@ -4859,10 +5212,8 @@ setPath(JsonbIterator **it, Datum *path_elems, case WJB_BEGIN_OBJECT: (void) pushJsonbValue(st, r, NULL); setPathObject(it, path_elems, path_nulls, path_len, st, level, - newval, v.val.object.nPairs, op_type); - r = JsonbIteratorNext(it, &v, true); - Assert(r == WJB_END_OBJECT); - res = pushJsonbValue(st, r, NULL); + newval, op_type); + res = pushJsonbValue(st, WJB_END_OBJECT, NULL); break; case WJB_ELEM: case WJB_VALUE: @@ -4878,109 +5229,118 @@ setPath(JsonbIterator **it, Datum *path_elems, } /* - * Object walker for setPath + * Copy object fields, but stop on the desired key if it is specified. + * + * True is returned if the key was found, otherwise false. */ -static void -setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, - int path_len, JsonbParseState **st, int level, - JsonbValue *newval, uint32 npairs, int op_type) +static bool +copyJsonbObject(JsonbParseState **st, JsonbIterator **it, const JsonbValue *key) { - int i; - JsonbValue k, - v; - bool done = false; - - if (level >= path_len || path_nulls[level]) - done = true; + JsonbIteratorToken r; + JsonbValue keybuf; - /* empty object is a special case for create */ - if ((npairs == 0) && (op_type & JB_PATH_CREATE_OR_INSERT) && - (level == path_len - 1)) + while ((r = JsonbIteratorNext(it, &keybuf, true)) == WJB_KEY) { - JsonbValue newkey; + JsonbValue val; - newkey.type = jbvString; - newkey.val.string.len = VARSIZE_ANY_EXHDR(path_elems[level]); - newkey.val.string.val = VARDATA_ANY(path_elems[level]); + if (key && + key->val.string.len == keybuf.val.string.len && + memcmp(key->val.string.val, keybuf.val.string.val, + key->val.string.len) == 0) + return true; /* stop, key is found */ - (void) pushJsonbValue(st, WJB_KEY, &newkey); - (void) pushJsonbValue(st, WJB_VALUE, newval); - } - - for (i = 0; i < npairs; i++) - { - JsonbIteratorToken r = JsonbIteratorNext(it, &k, true); + (void) pushJsonbValue(st, r, &keybuf); - Assert(r == WJB_KEY); + /* Copy value */ + r = JsonbIteratorNext(it, &val, false); + (void) pushJsonbValue(st, r, r < WJB_BEGIN_ARRAY ? &val : NULL); - if (!done && - k.val.string.len == VARSIZE_ANY_EXHDR(path_elems[level]) && - memcmp(k.val.string.val, VARDATA_ANY(path_elems[level]), - k.val.string.len) == 0) + if (r == WJB_BEGIN_ARRAY || r == WJB_BEGIN_OBJECT) { - if (level == path_len - 1) - { - /* - * called from jsonb_insert(), it forbids redefining an - * existing value - */ - if (op_type & (JB_PATH_INSERT_BEFORE | JB_PATH_INSERT_AFTER)) - ereport(ERROR, - (errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("cannot replace existing key"), - errhint("Try using the function jsonb_set " - "to replace key value."))); + int walking_level = 1; - r = JsonbIteratorNext(it, &v, true); /* skip value */ - if (!(op_type & JB_PATH_DELETE)) - { - (void) pushJsonbValue(st, WJB_KEY, &k); - (void) pushJsonbValue(st, WJB_VALUE, newval); - } - done = true; - } - else + while (walking_level != 0) { - (void) pushJsonbValue(st, r, &k); - setPath(it, path_elems, path_nulls, path_len, - st, level + 1, newval, op_type); + r = JsonbIteratorNext(it, &val, false); + + if (r == WJB_BEGIN_ARRAY || r == WJB_BEGIN_OBJECT) + ++walking_level; + if (r == WJB_END_ARRAY || r == WJB_END_OBJECT) + --walking_level; + + (void) pushJsonbValue(st, r, r < WJB_BEGIN_ARRAY ? &val : NULL); } } - else - { - if ((op_type & JB_PATH_CREATE_OR_INSERT) && !done && - level == path_len - 1 && i == npairs - 1) - { - JsonbValue newkey; + } - newkey.type = jbvString; - newkey.val.string.len = VARSIZE_ANY_EXHDR(path_elems[level]); - newkey.val.string.val = VARDATA_ANY(path_elems[level]); + Assert(r == WJB_END_OBJECT); - (void) pushJsonbValue(st, WJB_KEY, &newkey); - (void) pushJsonbValue(st, WJB_VALUE, newval); - } + return false; /* key was not found */ +} - (void) pushJsonbValue(st, r, &k); - r = JsonbIteratorNext(it, &v, false); - (void) pushJsonbValue(st, r, r < WJB_BEGIN_ARRAY ? &v : NULL); - if (r == WJB_BEGIN_ARRAY || r == WJB_BEGIN_OBJECT) - { - int walking_level = 1; +/* + * Object walker for setPath + */ +static void +setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, + int path_len, JsonbParseState **st, int level, + JsonbValue *newval, int op_type) +{ + JsonbValue *key, + keybuf, + val; - while (walking_level != 0) - { - r = JsonbIteratorNext(it, &v, false); + if (level >= path_len || path_nulls[level]) + key = NULL; + else + { + key = &keybuf; + key->type = jbvString; + key->val.string.val = VARDATA_ANY(path_elems[level]); + key->val.string.len = VARSIZE_ANY_EXHDR(path_elems[level]); + } - if (r == WJB_BEGIN_ARRAY || r == WJB_BEGIN_OBJECT) - ++walking_level; - if (r == WJB_END_ARRAY || r == WJB_END_OBJECT) - --walking_level; + /* Start copying object fields and stop on the desired key. */ + if (copyJsonbObject(st, it, key)) + { + /* The desired key was found. */ + if (level == path_len - 1) + { + /* + * called from jsonb_insert(), it forbids redefining an + * existing value + */ + if (op_type & (JB_PATH_INSERT_BEFORE | JB_PATH_INSERT_AFTER)) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("cannot replace existing key"), + errhint("Try using the function jsonb_set " + "to replace key value."))); - (void) pushJsonbValue(st, r, r < WJB_BEGIN_ARRAY ? &v : NULL); - } + (void) JsonbIteratorNext(it, &val, true); /* skip value */ + + if (!(op_type & JB_PATH_DELETE)) + { + (void) pushJsonbValue(st, WJB_KEY, key); + (void) pushJsonbValue(st, WJB_VALUE, newval); } } + else + { + (void) pushJsonbValue(st, WJB_KEY, key); + setPath(it, path_elems, path_nulls, path_len, st, level + 1, + newval, op_type); + } + + /* Copy the remaining fields. */ + (void) copyJsonbObject(st, it, NULL); + } + else if (key && (op_type & JB_PATH_CREATE_OR_INSERT) && + level == path_len - 1) + { + /* All fields were copied, but the desired key was not found. */ + (void) pushJsonbValue(st, WJB_KEY, key); + (void) pushJsonbValue(st, WJB_VALUE, newval); } } @@ -5000,18 +5360,14 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, /* pick correct index */ if (level < path_len && !path_nulls[level]) { - char *c = TextDatumGetCString(path_elems[level]); long lindex; - char *badp; - errno = 0; - lindex = strtol(c, &badp, 10); - if (errno != 0 || badp == c || *badp != '\0' || lindex > INT_MAX || - lindex < INT_MIN) + if (!jsonb_get_array_index_from_text(path_elems[level], &lindex)) ereport(ERROR, (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("path element at position %d is not an integer: \"%s\"", - level + 1, c))); + level + 1, TextDatumGetCString(path_elems[level])))); + idx = lindex; } else @@ -5113,14 +5469,12 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, Datum jsonb_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) { - return jsonb_get_element(DatumGetJsonbP(containerSource), - sbstate->upperindex, - sbstate->numupper, - &sbstate->resnull, - false); -} + JsonbValue *jbv = sbstate->privatedata; + Assert(!sbstate->isassignment); + return JsonbPGetDatum(JsonbValueToJsonb(jbv)); +} /* * Perform an actual data extraction or modification for the jsonb @@ -5130,19 +5484,51 @@ jsonb_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate) Datum jsonb_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) { - /* - * the original jsonb must be non-NULL, else we punt and return the - * original array. - */ + JsonbAssignState *astate = sbstate->privatedata; + JsonbSubscriptState *subscript = &astate->subscripts[sbstate->numupper - 1]; + JsonbValue *res = NULL; + JsonbValue *newval; + JsonbValue jbv; + JsonbIteratorToken tok; + + /* If the original jsonb is NULL, we will create a new container. */ if (sbstate->resnull) - return containerSource; + sbstate->resnull = false; - return jsonb_set_element(containerSource, - sbstate->upperindex, - sbstate->numupper, - sbstate->replacevalue, - sbstate->refelemtype, + /* Transform the new value to jsonb */ + newval = to_jsonb_worker(sbstate->replacevalue, sbstate->refelemtype, sbstate->replacenull); + + if (newval->type == jbvArray && newval->val.array.rawScalar) + *newval = newval->val.array.elems[0]; + + /* Push the new value */ + tok = subscript->is_array ? WJB_ELEM : WJB_VALUE; + res = pushJsonbValue(&astate->ps, tok, newval); + + /* Finish unclosed arrays/objects */ + for (; subscript >= astate->subscripts; subscript--) + { + /* + * If the element does exists, then all preceding subscripts must exist + * and the iterator may contain remaining elements. So we need to + * switch to copying from the iterator now. + */ + if (subscript[1].exists) + break; + + if (subscript->is_array && subscript->array_index < 0 && subscript->exists) + break; /* original elements are copied from the iterator */ + + tok = subscript->is_array ? WJB_END_ARRAY : WJB_END_OBJECT; + res = pushJsonbValue(&astate->ps, tok, NULL); + } + + /* Copy remaining elements from the iterator */ + while ((tok = JsonbIteratorNext(&astate->iter, &jbv, false)) != WJB_DONE) + res = pushJsonbValue(&astate->ps, tok, tok < WJB_BEGIN_ARRAY ? &jbv : NULL); + + return JsonbPGetDatum(JsonbValueToJsonb(res)); } /* @@ -5162,12 +5548,15 @@ Datum jsonb_subscript_handler(PG_FUNCTION_ARGS) { SubscriptRoutines *sbsroutines = (SubscriptRoutines *) - palloc(sizeof(SubscriptRoutines)); + palloc0(sizeof(SubscriptRoutines)); sbsroutines->prepare = jsonb_subscript_prepare; sbsroutines->validate = jsonb_subscript_validate; sbsroutines->fetch = jsonb_subscript_fetch; sbsroutines->assign = jsonb_subscript_assign; + sbsroutines->init = jsonb_subscript_init; + sbsroutines->step = jsonb_subscript_step; + sbsroutines->selectexpr = jsonb_subscript_selectexpr; PG_RETURN_POINTER(sbsroutines); } @@ -5190,19 +5579,37 @@ SubscriptingRef * jsonb_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, ParseState *pstate) { - List *upperIndexpr = NIL; - ListCell *l; + List *upperIndexpr = NIL; + ListCell *l; if (sbsref->reflowerindexpr != NIL) + { + Node *slice = NULL; + + /* Try to find first non-NULL lower subscript */ + foreach(l, sbsref->reflowerindexpr) + { + if (lfirst(l) != NULL) + { + slice = (Node *) lfirst(l); + break; + } + } + ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("jsonb subscript does not support slices"), - parser_errposition(pstate, exprLocation( - ((Node *) linitial(sbsref->reflowerindexpr)))))); + parser_errposition(pstate, exprLocation(slice)))); + } foreach(l, sbsref->refupperindexpr) { - Node *subexpr = (Node *) lfirst(l); + Node *subexpr = (Node *) lfirst(l); + Node *textexpr; + Node *intexpr; + Oid subexprType; + char typcategory; + bool typispreferred; if (subexpr == NULL) ereport(ERROR, @@ -5211,18 +5618,48 @@ jsonb_subscript_validate(bool isAssignment, SubscriptingRef *sbsref, parser_errposition(pstate, exprLocation( ((Node *) linitial(sbsref->refupperindexpr)))))); - subexpr = coerce_to_target_type(pstate, - subexpr, exprType(subexpr), - TEXTOID, -1, - COERCION_ASSIGNMENT, - COERCE_IMPLICIT_CAST, - -1); - if (subexpr == NULL) + subexprType = exprType(subexpr); + + textexpr = coerce_to_target_type(pstate, + subexpr, subexprType, + TEXTOID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + + if (textexpr == NULL) ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), - errmsg("jsonb subscript must have text type"), + errmsg("jsonb subscript must have %s type", "text"), parser_errposition(pstate, exprLocation(subexpr)))); + /* Try to coerce numeric types to int4 for array subscripting. */ + get_type_category_preferred(subexprType, &typcategory, &typispreferred); + + if (typcategory == TYPCATEGORY_NUMERIC) + { + intexpr = coerce_to_target_type(pstate, + subexpr, subexprType, + INT4OID, -1, + COERCION_ASSIGNMENT, + COERCE_IMPLICIT_CAST, + -1); + + if (intexpr && + (subexprType == INT4OID || + subexprType == INT2OID)) + textexpr = NULL; + } + else + intexpr = NULL; + + /* + * If int4 expression variant exists, create a list with both text and + * int4 variants. + */ + subexpr = textexpr && intexpr ? (Node *) list_make2(textexpr, intexpr) : + textexpr ? textexpr : intexpr; + upperIndexpr = lappend(upperIndexpr, subexpr); } diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index bc063061cf..d751343b4b 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -783,7 +783,7 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, else { v = getIthJsonbValueFromContainer(jb->val.binary.data, - (uint32) index); + (uint32) index, NULL); if (v == NULL) continue; diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h index 846b834f2d..68bfe7aec5 100644 --- a/src/include/executor/execExpr.h +++ b/src/include/executor/execExpr.h @@ -185,6 +185,12 @@ typedef enum ExprEvalOp */ EEOP_FIELDSTORE_FORM, + /* Init subscripting */ + EEOP_SBSREF_INIT, + + /* Select an expression for container subscript evaluation */ + EEOP_SBSREF_SELECTEXPR, + /* Process a container subscript; short-circuit expression to NULL if NULL */ EEOP_SBSREF_SUBSCRIPT, @@ -494,17 +500,30 @@ typedef struct ExprEvalStep int ncolumns; } fieldstore; + /* for EEOP_SBSREF_SELECTEXPR */ + struct + { + /* too big to have inline */ + struct SubscriptingRefState *state; + int off; /* 0-based index of this subscript */ + bool isupper; /* is it upper or lower subscript? */ + int nexprs; /* subscript expression count */ + Oid *exprtypes; /* type oids of subscript expression variants */ + int *jumpdones; /* jumps to expression variants */ + } sbsref_selectexpr; + /* for EEOP_SBSREF_SUBSCRIPT */ struct { /* too big to have inline */ struct SubscriptingRefState *state; + Oid typid; /* type oid of subscript */ int off; /* 0-based index of this subscript */ bool isupper; /* is it upper or lower subscript? */ int jumpdone; /* jump here on null */ } sbsref_subscript; - /* for EEOP_SBSREF_OLD / ASSIGN / FETCH */ + /* for EEOP_SBSREF_OLD / ASSIGN / FETCH / INIT */ struct { /* too big to have inline */ @@ -651,11 +670,13 @@ typedef struct SubscriptingRefState /* at runtime, extracted subscript datums get stored in upperindex[] */ int numupper; bool upperprovided[MAX_SUBSCRIPT_DEPTH]; + Oid uppertypid[MAX_SUBSCRIPT_DEPTH]; Datum upperindex[MAX_SUBSCRIPT_DEPTH]; /* similarly for lower indexes, if any */ int numlower; bool lowerprovided[MAX_SUBSCRIPT_DEPTH]; + Oid lowertypid[MAX_SUBSCRIPT_DEPTH]; Datum lowerindex[MAX_SUBSCRIPT_DEPTH]; /* subscript expressions get evaluated into here */ @@ -672,6 +693,7 @@ typedef struct SubscriptingRefState bool resnull; struct SubscriptRoutines *sbsroutines; + void *privatedata; } SubscriptingRefState; @@ -716,6 +738,8 @@ extern void ExecEvalFieldStoreDeForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext); extern void ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext); +extern void ExecEvalSubscriptingRefInit(ExprState *state, ExprEvalStep *op); +extern int ExecEvalSubscriptingRefSelect(ExprState *state, ExprEvalStep *op); extern bool ExecEvalSubscriptingRef(ExprState *state, ExprEvalStep *op); extern void ExecEvalSubscriptingRefFetch(ExprState *state, ExprEvalStep *op); extern void ExecEvalSubscriptingRefOld(ExprState *state, ExprEvalStep *op); diff --git a/src/include/nodes/subscripting.h b/src/include/nodes/subscripting.h index 1800d5ecf5..d752bf6490 100644 --- a/src/include/nodes/subscripting.h +++ b/src/include/nodes/subscripting.h @@ -29,13 +29,25 @@ typedef Datum (*SubscriptingFetch) (Datum source, struct SubscriptingRefState *s typedef Datum (*SubscriptingAssign) (Datum source, struct SubscriptingRefState *sbrsefstate); +typedef void (*SubscriptingInit) (struct SubscriptingRefState *sbrefstate, + Datum source, bool isnull); + +typedef int (*SubscriptingSelectExpr) (struct SubscriptingRefState *sbsreftate, + int subscriptNum, Oid subscriptType, + Oid *subscriptExprTypes, int nexprs); + +typedef bool (*SubscriptingStep) (struct SubscriptingRefState *sbrefstate, + int subscriptNum, bool isupper); + typedef struct SubscriptRoutines { SubscriptingPrepare prepare; SubscriptingValidate validate; SubscriptingFetch fetch; SubscriptingAssign assign; - + SubscriptingInit init; + SubscriptingStep step; + SubscriptingSelectExpr selectexpr; } SubscriptRoutines; diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h index 6e3b75d56a..bbc962e559 100644 --- a/src/include/utils/jsonb.h +++ b/src/include/utils/jsonb.h @@ -386,7 +386,7 @@ extern JsonbValue *getKeyJsonValueFromContainer(JsonbContainer *container, const char *keyVal, int keyLen, JsonbValue *res); extern JsonbValue *getIthJsonbValueFromContainer(JsonbContainer *sheader, - uint32 i); + uint32 i, JsonbValue *result); extern JsonbValue *pushJsonbValue(JsonbParseState **pstate, JsonbIteratorToken seq, JsonbValue *jbval); extern JsonbIterator *JsonbIteratorInit(JsonbContainer *container); diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out index 04a146a7d0..16ffdcecf9 100644 --- a/src/test/regress/expected/jsonb.out +++ b/src/test/regress/expected/jsonb.out @@ -4758,12 +4758,48 @@ select ('[1, "2", null]'::jsonb)['1']; "2" (1 row) -select ('[1, "2", null]'::jsonb)[1.0]; +select ('[1, "2", null]'::jsonb)['1.0']; jsonb ------- (1 row) +select ('[1, "2", null]'::jsonb)[1.0]; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1.4]; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1.5]; + jsonb +------- + null +(1 row) + +select ('[1, "2", null]'::jsonb)[1.6]; + jsonb +------- + null +(1 row) + +select ('[1, "2", null]'::jsonb)[0.6]; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)['0.3'::numeric]; + jsonb +------- + 1 +(1 row) + select ('[1, "2", null]'::jsonb)[2]; jsonb ------- @@ -4848,6 +4884,54 @@ select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1 "ccc" (1 row) +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1]; + jsonb +------- + "a" +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.0]; + jsonb +------- + "b" +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.1]; + jsonb +------- + +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.2]; + jsonb +------- + "c" +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1']; + jsonb +------- + "a" +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.0']; + jsonb +------- + "b" +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.1']; + jsonb +------- + +(1 row) + +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.2']; + jsonb +------- + "c" +(1 row) + create TEMP TABLE test_jsonb_subscript ( id int, test_json jsonb @@ -4900,11 +4984,75 @@ select * from test_jsonb_subscript; 2 | {"a": [1, 2, 3], "key": "value"} (2 rows) +-- replace element in string subscript +update test_jsonb_subscript set test_json['a']['3'] = '4'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------- + 1 | {"a": [1, 2, 3, 4]} + 2 | {"a": [1, 2, 3, 4], "key": "value"} +(2 rows) + +-- bad array subscripts +update test_jsonb_subscript set test_json['a']['a'] = '5'::jsonb; +ERROR: jsonb array subscript is not an integer: "a" +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------- + 1 | {"a": [1, 2, 3, 4]} + 2 | {"a": [1, 2, 3, 4], "key": "value"} +(2 rows) + +update test_jsonb_subscript set test_json['a'][0][1] = '5'::jsonb; +ERROR: cannot assign to subscript of scalar jsonb +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------- + 1 | {"a": [1, 2, 3, 4]} + 2 | {"a": [1, 2, 3, 4], "key": "value"} +(2 rows) + +-- append element to array +update test_jsonb_subscript set test_json['a'][2.9] = '"4"'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------------- + 1 | {"a": [1, 2, 3, "4"]} + 2 | {"a": [1, 2, 3, "4"], "key": "value"} +(2 rows) + +-- append element to array with a gap filled with nulls +update test_jsonb_subscript set test_json['a'][7] = '8'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------ + 1 | {"a": [1, 2, 3, "4", 8]} + 2 | {"a": [1, 2, 3, "4", 8], "key": "value"} +(2 rows) + +-- replace element in array using negative subscript +update test_jsonb_subscript set test_json['a'][-4] = '5'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------ + 1 | {"a": [1, 5, 3, "4", 8]} + 2 | {"a": [1, 5, 3, "4", 8], "key": "value"} +(2 rows) + +-- prepend element to array using negative subscript with a gap filled with nulls +update test_jsonb_subscript set test_json['a'][-10] = '6'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------------------- + 1 | {"a": [6, 1, 5, 3, "4", 8]} + 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value"} +(2 rows) + -- use jsonb subscription in where clause select * from test_jsonb_subscript where test_json['key'] = '"value"'; - id | test_json -----+---------------------------------- - 2 | {"a": [1, 2, 3], "key": "value"} + id | test_json +----+--------------------------------------------- + 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value"} (1 row) select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; @@ -4922,12 +5070,81 @@ update test_jsonb_subscript set test_json[NULL] = 1; ERROR: subscript in assignment must not be null update test_jsonb_subscript set test_json['another_key'] = NULL; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------------------- - 1 | {"a": [1, 2, 3], "another_key": null} - 2 | {"a": [1, 2, 3], "key": "value", "another_key": null} + id | test_json +----+------------------------------------------------------------------ + 1 | {"a": [6, 1, 5, 3, "4", 8], "another_key": null} + 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value", "another_key": null} (2 rows) +-- create a path +delete from test_jsonb_subscript; +insert into test_jsonb_subscript values (0, NULL); +update test_jsonb_subscript set test_json['a'] = '1'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+----------- + 0 | {"a": 1} +(1 row) + +update test_jsonb_subscript set test_json['b'][0] = '2'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+-------------------- + 0 | {"a": 1, "b": [2]} +(1 row) + +update test_jsonb_subscript set test_json['c'][0]['a'] = '3'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------- + 0 | {"a": 1, "b": [2], "c": [{"a": 3}]} +(1 row) + +update test_jsonb_subscript set test_json['d'][0]['a'][3] = '4'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+-------------------------------------------------------- + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4]}]} +(1 row) + +update test_jsonb_subscript set test_json['d'][0]['c'][-3] = '5'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------------------------------ + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "c": [5]}]} +(1 row) + +update test_jsonb_subscript set test_json['d'][0]['b']['x'] = '6'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------------------------------------------------------- + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "b": {"x": 6}, "c": [5]}]} +(1 row) + +update test_jsonb_subscript set test_json['e']['y'] = '7'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------------------------------------------------------------ + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "b": {"x": 6}, "c": [5]}], "e": {"y": 7}} +(1 row) + +-- updating of scalar's subscripts +update test_jsonb_subscript set test_json = '1'; +update test_jsonb_subscript set test_json['a'] = '1'::jsonb; +ERROR: cannot assign to subscript of scalar jsonb +update test_jsonb_subscript set test_json[0] = '1'::jsonb; +ERROR: cannot assign to subscript of scalar jsonb +update test_jsonb_subscript set test_json = '{"a": 1}'; +update test_jsonb_subscript set test_json['a']['a'] = '1'::jsonb; +ERROR: cannot assign to subscript of scalar jsonb +update test_jsonb_subscript set test_json['a'][0] = '1'::jsonb; +ERROR: cannot assign to subscript of scalar jsonb +select * from test_jsonb_subscript; + id | test_json +----+----------- + 0 | {"a": 1} +(1 row) + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); to_tsvector diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql index 12541e7e50..997189f57f 100644 --- a/src/test/regress/sql/jsonb.sql +++ b/src/test/regress/sql/jsonb.sql @@ -1214,7 +1214,13 @@ select ('{"a": 1}'::jsonb)[NULL]; select ('[1, "2", null]'::jsonb)['a']; select ('[1, "2", null]'::jsonb)[0]; select ('[1, "2", null]'::jsonb)['1']; +select ('[1, "2", null]'::jsonb)['1.0']; select ('[1, "2", null]'::jsonb)[1.0]; +select ('[1, "2", null]'::jsonb)[1.4]; +select ('[1, "2", null]'::jsonb)[1.5]; +select ('[1, "2", null]'::jsonb)[1.6]; +select ('[1, "2", null]'::jsonb)[0.6]; +select ('[1, "2", null]'::jsonb)['0.3'::numeric]; select ('[1, "2", null]'::jsonb)[2]; select ('[1, "2", null]'::jsonb)[3]; select ('[1, "2", null]'::jsonb)[-2]; @@ -1229,6 +1235,15 @@ select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1' select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']['a3']; select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1']; select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1'][2]; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1]; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.0]; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.1]; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)[1.2]; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1']; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.0']; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.1']; +select ('{"1": "a", "1.0": "b", "1.2": "c"}'::jsonb)['1.2']; + create TEMP TABLE test_jsonb_subscript ( id int, @@ -1259,6 +1274,35 @@ select * from test_jsonb_subscript; update test_jsonb_subscript set test_json['a'] = '[1, 2, 3]'::jsonb; select * from test_jsonb_subscript; +-- replace element in string subscript +update test_jsonb_subscript set test_json['a']['3'] = '4'::jsonb; +select * from test_jsonb_subscript; + +-- bad array subscripts +update test_jsonb_subscript set test_json['a']['a'] = '5'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['a'][0][1] = '5'::jsonb; +select * from test_jsonb_subscript; + + +-- append element to array +update test_jsonb_subscript set test_json['a'][2.9] = '"4"'::jsonb; +select * from test_jsonb_subscript; + +-- append element to array with a gap filled with nulls +update test_jsonb_subscript set test_json['a'][7] = '8'::jsonb; +select * from test_jsonb_subscript; + +-- replace element in array using negative subscript +update test_jsonb_subscript set test_json['a'][-4] = '5'::jsonb; +select * from test_jsonb_subscript; + +-- prepend element to array using negative subscript with a gap filled with nulls +update test_jsonb_subscript set test_json['a'][-10] = '6'::jsonb; +select * from test_jsonb_subscript; + + -- use jsonb subscription in where clause select * from test_jsonb_subscript where test_json['key'] = '"value"'; select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; @@ -1269,6 +1313,42 @@ update test_jsonb_subscript set test_json[NULL] = 1; update test_jsonb_subscript set test_json['another_key'] = NULL; select * from test_jsonb_subscript; +-- create a path +delete from test_jsonb_subscript; +insert into test_jsonb_subscript values (0, NULL); + +update test_jsonb_subscript set test_json['a'] = '1'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['b'][0] = '2'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['c'][0]['a'] = '3'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['d'][0]['a'][3] = '4'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['d'][0]['c'][-3] = '5'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['d'][0]['b']['x'] = '6'::jsonb; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json['e']['y'] = '7'::jsonb; +select * from test_jsonb_subscript; + +-- updating of scalar's subscripts +update test_jsonb_subscript set test_json = '1'; +update test_jsonb_subscript set test_json['a'] = '1'::jsonb; +update test_jsonb_subscript set test_json[0] = '1'::jsonb; + +update test_jsonb_subscript set test_json = '{"a": 1}'; +update test_jsonb_subscript set test_json['a']['a'] = '1'::jsonb; +update test_jsonb_subscript set test_json['a'][0] = '1'::jsonb; + +select * from test_jsonb_subscript; + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); -- 2.21.0 --g5st7yssq7xavhfb Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0006-Filling-gaps-in-jsonb-arrays.patch" From be53c66e5641bcf581f79df5a65603eb7218ae07 Mon Sep 17 00:00:00 2001 From: Dmitrii Dolgov <9erthalion6@gmail.com> Date: Thu, 19 Dec 2019 14:13:35 +0100 Subject: [PATCH v30 6/6] Filling gaps in jsonb arrays Appending or prepending array elements on the specified position, gaps filled with nulls (JavaScript has similar behavior) Author: Nikita Glukhov --- src/backend/utils/adt/jsonfuncs.c | 26 +++++++++++- src/test/regress/expected/jsonb.out | 62 ++++++++++++++--------------- 2 files changed, 55 insertions(+), 33 deletions(-) diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c index 82d4a98e7e..e86d3918dd 100644 --- a/src/backend/utils/adt/jsonfuncs.c +++ b/src/backend/utils/adt/jsonfuncs.c @@ -1753,6 +1753,17 @@ jsonb_subscript_apply(JsonbValue *jbv, Datum subscriptVal, Oid subscriptTypid, } } +static void +push_null_elements(JsonbParseState **ps, int num) +{ + JsonbValue null; + + null.type = jbvNull; + + while (num-- > 0) + pushJsonbValue(ps, WJB_ELEM, &null); +} + /* Perfrom one subscript assignment step */ static void jsonb_subscript_step_assignment(SubscriptingRefState *sbstate, Datum value, @@ -1809,6 +1820,10 @@ jsonb_subscript_step_assignment(SubscriptingRefState *sbstate, Datum value, JsonbIteratorNext(&astate->iter, &jbv, last) != WJB_END_ARRAY) subscript[1].exists = true; } + + /* Fill the gap before the new element with nulls */ + if (i < index) + push_null_elements(&astate->ps, index - i); } else { @@ -5517,8 +5532,15 @@ jsonb_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate) if (subscript[1].exists) break; - if (subscript->is_array && subscript->array_index < 0 && subscript->exists) - break; /* original elements are copied from the iterator */ + if (subscript->is_array && subscript->array_index < 0) + { + /* Fill the gap between prepended element and 0th element */ + if (subscript->array_index < -1) + push_null_elements(&astate->ps, -1 - subscript->array_index); + + if (subscript->exists) + break; /* original elements are copied from the iterator */ + } tok = subscript->is_array ? WJB_END_ARRAY : WJB_END_OBJECT; res = pushJsonbValue(&astate->ps, tok, NULL); diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out index 16ffdcecf9..89b1452a75 100644 --- a/src/test/regress/expected/jsonb.out +++ b/src/test/regress/expected/jsonb.out @@ -5024,35 +5024,35 @@ select * from test_jsonb_subscript; -- append element to array with a gap filled with nulls update test_jsonb_subscript set test_json['a'][7] = '8'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------ - 1 | {"a": [1, 2, 3, "4", 8]} - 2 | {"a": [1, 2, 3, "4", 8], "key": "value"} + id | test_json +----+------------------------------------------------------------ + 1 | {"a": [1, 2, 3, "4", null, null, null, 8]} + 2 | {"a": [1, 2, 3, "4", null, null, null, 8], "key": "value"} (2 rows) -- replace element in array using negative subscript update test_jsonb_subscript set test_json['a'][-4] = '5'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------ - 1 | {"a": [1, 5, 3, "4", 8]} - 2 | {"a": [1, 5, 3, "4", 8], "key": "value"} + id | test_json +----+--------------------------------------------------------- + 1 | {"a": [1, 2, 3, "4", 5, null, null, 8]} + 2 | {"a": [1, 2, 3, "4", 5, null, null, 8], "key": "value"} (2 rows) -- prepend element to array using negative subscript with a gap filled with nulls update test_jsonb_subscript set test_json['a'][-10] = '6'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+--------------------------------------------- - 1 | {"a": [6, 1, 5, 3, "4", 8]} - 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value"} + id | test_json +----+------------------------------------------------------------------ + 1 | {"a": [6, null, 1, 2, 3, "4", 5, null, null, 8]} + 2 | {"a": [6, null, 1, 2, 3, "4", 5, null, null, 8], "key": "value"} (2 rows) -- use jsonb subscription in where clause select * from test_jsonb_subscript where test_json['key'] = '"value"'; - id | test_json -----+--------------------------------------------- - 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value"} + id | test_json +----+------------------------------------------------------------------ + 2 | {"a": [6, null, 1, 2, 3, "4", 5, null, null, 8], "key": "value"} (1 row) select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; @@ -5070,10 +5070,10 @@ update test_jsonb_subscript set test_json[NULL] = 1; ERROR: subscript in assignment must not be null update test_jsonb_subscript set test_json['another_key'] = NULL; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------------------------------ - 1 | {"a": [6, 1, 5, 3, "4", 8], "another_key": null} - 2 | {"a": [6, 1, 5, 3, "4", 8], "key": "value", "another_key": null} + id | test_json +----+--------------------------------------------------------------------------------------- + 1 | {"a": [6, null, 1, 2, 3, "4", 5, null, null, 8], "another_key": null} + 2 | {"a": [6, null, 1, 2, 3, "4", 5, null, null, 8], "key": "value", "another_key": null} (2 rows) -- create a path @@ -5102,30 +5102,30 @@ select * from test_jsonb_subscript; update test_jsonb_subscript set test_json['d'][0]['a'][3] = '4'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+-------------------------------------------------------- - 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4]}]} + id | test_json +----+-------------------------------------------------------------------------- + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [null, null, null, 4]}]} (1 row) update test_jsonb_subscript set test_json['d'][0]['c'][-3] = '5'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------------------------------ - 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "c": [5]}]} + id | test_json +----+------------------------------------------------------------------------------------------------ + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [null, null, null, 4], "c": [5, null, null]}]} (1 row) update test_jsonb_subscript set test_json['d'][0]['b']['x'] = '6'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+--------------------------------------------------------------------------------- - 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "b": {"x": 6}, "c": [5]}]} + id | test_json +----+--------------------------------------------------------------------------------------------------------------- + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [null, null, null, 4], "b": {"x": 6}, "c": [5, null, null]}]} (1 row) update test_jsonb_subscript set test_json['e']['y'] = '7'::jsonb; select * from test_jsonb_subscript; - id | test_json -----+------------------------------------------------------------------------------------------------ - 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [4], "b": {"x": 6}, "c": [5]}], "e": {"y": 7}} + id | test_json +----+------------------------------------------------------------------------------------------------------------------------------ + 0 | {"a": 1, "b": [2], "c": [{"a": 3}], "d": [{"a": [null, null, null, 4], "b": {"x": 6}, "c": [5, null, null]}], "e": {"y": 7}} (1 row) -- updating of scalar's subscripts -- 2.21.0 --g5st7yssq7xavhfb--