public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v43 1/2] Subscripting for jsonb 2+ messages / 2 participants [nested] [flat]
* [PATCH v43 1/2] Subscripting for jsonb @ 2020-12-18 16:19 Dmitrii Dolgov <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Dmitrii Dolgov @ 2020-12-18 16:19 UTC (permalink / raw) Subscripting implementation for jsonb. It does not support slices, does not have a limit for number of subscripts and for assignment expects a replace value to be of jsonb type. There is also one functional difference in assignment via subscripting from jsonb_set, when an original jsonb container is NULL, subscripting replaces it with an empty jsonb and proceed with assignment. For the sake of code reuse, some parts of jsonb functionality were rearranged to allow use the same functions for jsonb_set and assign subscripting operation. The original idea belongs to Oleg Bartunov. Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule --- doc/src/sgml/json.sgml | 48 ++++ src/backend/utils/adt/Makefile | 1 + src/backend/utils/adt/jsonb_util.c | 76 ++++- src/backend/utils/adt/jsonbsubs.c | 413 ++++++++++++++++++++++++++++ src/backend/utils/adt/jsonfuncs.c | 180 ++++++------ src/include/catalog/pg_proc.dat | 4 + src/include/catalog/pg_type.dat | 3 +- src/include/utils/jsonb.h | 6 +- src/test/regress/expected/jsonb.out | 272 +++++++++++++++++- src/test/regress/sql/jsonb.sql | 84 +++++- 10 files changed, 982 insertions(+), 105 deletions(-) create mode 100644 src/backend/utils/adt/jsonbsubs.c diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml index 5b9a5557a4..100d1a60f4 100644 --- a/doc/src/sgml/json.sgml +++ b/doc/src/sgml/json.sgml @@ -602,6 +602,54 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu </para> </sect2> + <sect2 id="jsonb-subscripting"> + <title><type>jsonb</type> Subscripting</title> + <para> + <type>jsonb</type> data type supports array-style subscripting expressions + to extract or update particular elements. It's possible to use multiple + subscripting expressions to extract nested values. In this case, a chain of + subscripting expressions follows the same rules as the + <literal>path</literal> argument in <literal>jsonb_set</literal> function, + e.g. in case of arrays it is a 0-based operation or that negative integers + that appear in <literal>path</literal> count from the end of JSON arrays. + The result of subscripting expressions is always jsonb data type. An + example of subscripting syntax: +<programlisting> +-- Extract value by key +SELECT ('{"a": 1}'::jsonb)['a']; + +-- Extract nested value by key path +SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c']; + +-- Extract element by index +SELECT ('[1, "2", null]'::jsonb)[1]; + +-- Update value by key, note the single quotes - the assigned value +-- needs to be of jsonb type as well +UPDATE table_name SET jsonb_field['key'] = '1'; + +-- Select records using where clause with subscripting. Since the result of +-- subscripting is jsonb and we basically want to compare two jsonb objects, we +-- need to put the value in double quotes to be able to convert it to jsonb. +SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"'; +</programlisting> + + Subscripting for <type>jsonb</type> does not support slice expressions, + even if it contains an array. + + In case if source <type>jsonb</type> is <literal>NULL</literal>, assignment + via subscripting will proceed as if it was an empty JSON object: +<programlisting> +-- If jsonb_field here is NULL, the result is {"a": 1} +UPDATE table_name SET jsonb_field['a'] = '1'; + +-- If jsonb_field here is NULL, the result is [1] +UPDATE table_name SET jsonb_field[0] = '1'; +</programlisting> + + </para> + </sect2> + <sect2> <title>Transforms</title> diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile index 82732146d3..279ff15ade 100644 --- a/src/backend/utils/adt/Makefile +++ b/src/backend/utils/adt/Makefile @@ -50,6 +50,7 @@ OBJS = \ jsonb_op.o \ jsonb_util.o \ jsonfuncs.o \ + jsonbsubs.o \ jsonpath.o \ jsonpath_exec.o \ jsonpath_gram.o \ diff --git a/src/backend/utils/adt/jsonb_util.c b/src/backend/utils/adt/jsonb_util.c index 4eeffa1424..41a1c1f9bb 100644 --- a/src/backend/utils/adt/jsonb_util.c +++ b/src/backend/utils/adt/jsonb_util.c @@ -68,18 +68,29 @@ static JsonbValue *pushJsonbValueScalar(JsonbParseState **pstate, JsonbIteratorToken seq, JsonbValue *scalarVal); +JsonbValue * +JsonbToJsonbValue(Jsonb *jsonb) +{ + JsonbValue *val = (JsonbValue *) palloc(sizeof(JsonbValue)); + + val->type = jbvBinary; + val->val.binary.data = &jsonb->root; + val->val.binary.len = VARSIZE(jsonb) - VARHDRSZ; + + return val; +} + /* * Turn an in-memory JsonbValue into a Jsonb for on-disk storage. * - * There isn't a JsonbToJsonbValue(), because generally we find it more - * convenient to directly iterate through the Jsonb representation and only - * really convert nested scalar values. JsonbIteratorNext() does this, so that - * clients of the iteration code don't have to directly deal with the binary - * representation (JsonbDeepContains() is a notable exception, although all - * exceptions are internal to this module). In general, functions that accept - * a JsonbValue argument are concerned with the manipulation of scalar values, - * or simple containers of scalar values, where it would be inconvenient to - * deal with a great amount of other state. + * Generally we find it more convenient to directly iterate through the Jsonb + * representation and only really convert nested scalar values. + * JsonbIteratorNext() does this, so that clients of the iteration code don't + * have to directly deal with the binary representation (JsonbDeepContains() is + * a notable exception, although all exceptions are internal to this module). + * In general, functions that accept a JsonbValue argument are concerned with + * the manipulation of scalar values, or simple containers of scalar values, + * where it would be inconvenient to deal with a great amount of other state. */ Jsonb * JsonbValueToJsonb(JsonbValue *val) @@ -563,6 +574,30 @@ pushJsonbValue(JsonbParseState **pstate, JsonbIteratorToken seq, JsonbValue *res = NULL; JsonbValue v; JsonbIteratorToken tok; + int i; + + if (jbval && (seq == WJB_ELEM || seq == WJB_VALUE) && jbval->type == jbvObject) + { + pushJsonbValue(pstate, WJB_BEGIN_OBJECT, NULL); + for (i = 0; i < jbval->val.object.nPairs; i++) + { + pushJsonbValue(pstate, WJB_KEY, &jbval->val.object.pairs[i].key); + pushJsonbValue(pstate, WJB_VALUE, &jbval->val.object.pairs[i].value); + } + + return pushJsonbValue(pstate, WJB_END_OBJECT, NULL); + } + + if (jbval && (seq == WJB_ELEM || seq == WJB_VALUE) && jbval->type == jbvArray) + { + pushJsonbValue(pstate, WJB_BEGIN_ARRAY, NULL); + for (i = 0; i < jbval->val.array.nElems; i++) + { + pushJsonbValue(pstate, WJB_ELEM, &jbval->val.array.elems[i]); + } + + return pushJsonbValue(pstate, WJB_END_ARRAY, NULL); + } if (!jbval || (seq != WJB_ELEM && seq != WJB_VALUE) || jbval->type != jbvBinary) @@ -573,9 +608,30 @@ pushJsonbValue(JsonbParseState **pstate, JsonbIteratorToken seq, /* unpack the binary and add each piece to the pstate */ it = JsonbIteratorInit(jbval->val.binary.data); + + if ((jbval->val.binary.data->header & JB_FSCALAR) && *pstate) + { + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_BEGIN_ARRAY); + Assert(v.type == jbvArray && v.val.array.rawScalar); + + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_ELEM); + + res = pushJsonbValueScalar(pstate, seq, &v); + + tok = JsonbIteratorNext(&it, &v, true); + Assert(tok == WJB_END_ARRAY); + Assert(it == NULL); + + return res; + } + while ((tok = JsonbIteratorNext(&it, &v, false)) != WJB_DONE) res = pushJsonbValueScalar(pstate, tok, - tok < WJB_BEGIN_ARRAY ? &v : NULL); + tok < WJB_BEGIN_ARRAY || + (tok == WJB_BEGIN_ARRAY && + v.val.array.rawScalar) ? &v : NULL); return res; } diff --git a/src/backend/utils/adt/jsonbsubs.c b/src/backend/utils/adt/jsonbsubs.c new file mode 100644 index 0000000000..306c37b5a6 --- /dev/null +++ b/src/backend/utils/adt/jsonbsubs.c @@ -0,0 +1,413 @@ +/*------------------------------------------------------------------------- + * + * jsonbsubs.c + * Subscripting support functions for jsonb. + * + * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/utils/adt/jsonbsubs.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "executor/execExpr.h" +#include "nodes/makefuncs.h" +#include "nodes/nodeFuncs.h" +#include "nodes/subscripting.h" +#include "parser/parse_coerce.h" +#include "parser/parse_expr.h" +#include "utils/jsonb.h" +#include "utils/jsonfuncs.h" +#include "utils/builtins.h" +#include "utils/lsyscache.h" + + +/* SubscriptingRefState.workspace for jsonb subscripting execution */ +typedef struct JsonbSubWorkspace +{ + bool expectArray; /* jsonb root is expected to be an array */ + Oid *indexOid; /* OID of coerced subscript expression, + could be only integer or text */ + Datum *index; /* Subscript values in Datum format */ +} JsonbSubWorkspace; + + +/* + * Finish parse analysis of a SubscriptingRef expression for a jsonb. + * + * Transform the subscript expressions, coerce them to text, + * and determine the result type of the SubscriptingRef node. + */ +static void +jsonb_subscript_transform(SubscriptingRef *sbsref, + List *indirection, + ParseState *pstate, + bool isSlice, + bool isAssignment) +{ + List *upperIndexpr = NIL; + ListCell *idx; + + /* + * Transform and convert the subscript expressions. Jsonb subscripting does + * not support slices, look only and the upper index. + */ + foreach(idx, indirection) + { + A_Indices *ai = lfirst_node(A_Indices, idx); + Node *subExpr; + + if (isSlice) + { + Node *expr = ai->uidx ? ai->uidx : ai->lidx; + + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript does not support slices"), + parser_errposition(pstate, exprLocation(expr)))); + } + + if (ai->uidx) + { + Oid subExprType = InvalidOid, + targetType = UNKNOWNOID; + + subExpr = transformExpr(pstate, ai->uidx, pstate->p_expr_kind); + subExprType = exprType(subExpr); + + if (subExprType != UNKNOWNOID) + { + Oid targets[2] = {INT4OID, TEXTOID}; + + /* + * Jsonb can handle multiple subscript types, but cases when a + * subscript could be coerced to multiple target types must be + * avoided, similar to overloaded functions. It could be + * possibly extend with jsonpath in the future. + */ + for (int i = 0; i < 2; i++) + { + if (can_coerce_type(1, &subExprType, &targets[i], COERCION_IMPLICIT)) + { + /* + * One type has already succeeded, it means there are + * two coercion targets possible, failure. + */ + if (targetType != UNKNOWNOID) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("subscript type is not supported"), + errhint("Jsonb subscript must be coerced " + "only to one type, integer or text."), + parser_errposition(pstate, exprLocation(subExpr)))); + + targetType = targets[i]; + } + } + + /* + * No suitable types were found, failure. + */ + if (targetType == UNKNOWNOID) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("subscript type is not supported"), + errhint("Jsonb subscript must be coercet to either integer or text"), + parser_errposition(pstate, exprLocation(subExpr)))); + } + else + targetType = TEXTOID; + + /* + * We known from can_coerce_type that coercion will succeed, so + * coerce_type could be used. Note the implicit coercion context, + * which is required to handle subscripts of different types, + * similar to overloaded functions. + */ + subExpr = coerce_type(pstate, + subExpr, subExprType, + targetType, -1, + COERCION_IMPLICIT, + COERCE_IMPLICIT_CAST, + -1); + if (subExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript must have text type"), + parser_errposition(pstate, exprLocation(subExpr)))); + } + else + { + /* + * Slice with omitted upper bound. Should not happen as we already + * errored out on slice earlier, but handle this just in case. + */ + Assert(isSlice && ai->is_slice); + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("jsonb subscript does not support slices"), + parser_errposition(pstate, exprLocation(ai->uidx)))); + } + + upperIndexpr = lappend(upperIndexpr, subExpr); + } + + /* store the transformed lists into the SubscriptRef node */ + sbsref->refupperindexpr = upperIndexpr; + sbsref->reflowerindexpr = NIL; + + /* Determine the result type of the subscripting operation; always jsonb */ + sbsref->refrestype = JSONBOID; + sbsref->reftypmod = -1; +} + +/* + * During execution, process the subscripts in a SubscriptingRef expression. + * + * The subscript expressions are already evaluated in Datum form in the + * SubscriptingRefState's arrays. Check and convert them as necessary. + * + * If any subscript is NULL, we throw error in assignment cases, or in fetch + * cases set result to NULL and return false (instructing caller to skip the + * rest of the SubscriptingRef sequence). + */ +static bool +jsonb_subscript_check_subscripts(ExprState *state, + ExprEvalStep *op, + ExprContext *econtext) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref_subscript.state; + JsonbSubWorkspace *workspace = (JsonbSubWorkspace *) sbsrefstate->workspace; + + /* + * In case if the first subscript is an integer, the source jsonb is + * expected to be an array. This information is not used directly, all such + * cases are handled within corresponding jsonb assign functions. But if + * the source jsonb is NULL the expected type will be used to construct an + * empty source. + */ + if (sbsrefstate->numupper > 0 && sbsrefstate->upperprovided[0] && + !sbsrefstate->upperindexnull[0] && workspace->indexOid[0] == INT4OID) + workspace->expectArray = true; + + /* Process upper subscripts */ + for (int i = 0; i < sbsrefstate->numupper; i++) + { + if (sbsrefstate->upperprovided[i]) + { + /* If any index expr yields NULL, result is NULL or error */ + if (sbsrefstate->upperindexnull[i]) + { + if (sbsrefstate->isassignment) + ereport(ERROR, + (errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED), + errmsg("jsonb subscript in assignment must not be null"))); + *op->resnull = true; + return false; + } + + /* + * For jsonb fetch and assign functions we need to provide path in + * text format. Convert if it's not already text. + */ + if (workspace->indexOid[i] == INT4OID) + { + Datum datum = sbsrefstate->upperindex[i]; + char *cs = DatumGetCString(DirectFunctionCall1(int4out, datum)); + workspace->index[i] = CStringGetTextDatum(cs); + } + else + workspace->index[i] = sbsrefstate->upperindex[i]; + } + } + + return true; +} + +/* + * Evaluate SubscriptingRef fetch for a jsonb element. + * + * Source container is in step's result variable (it's known not NULL, since + * we set fetch_strict to true). + */ +static void +jsonb_subscript_fetch(ExprState *state, + ExprEvalStep *op, + ExprContext *econtext) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + JsonbSubWorkspace *workspace = (JsonbSubWorkspace *) sbsrefstate->workspace; + Jsonb *jsonbSource; + + /* Should not get here if source jsonb (or any subscript) is null */ + Assert(!(*op->resnull)); + + jsonbSource = DatumGetJsonbP(*op->resvalue); + *op->resvalue = jsonb_get_element(jsonbSource, + workspace->index, + sbsrefstate->numupper, + op->resnull, + false); +} + +/* + * Evaluate SubscriptingRef assignment for a jsonb element assignment. + * + * Input container (possibly null) is in result area, replacement value is in + * SubscriptingRefState's replacevalue/replacenull. + */ +static void +jsonb_subscript_assign(ExprState *state, + ExprEvalStep *op, + ExprContext *econtext) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + JsonbSubWorkspace *workspace = (JsonbSubWorkspace *) sbsrefstate->workspace; + Jsonb *jsonbSource; + JsonbValue *replacevalue; + + if (sbsrefstate->replacenull) + { + replacevalue = (JsonbValue *) palloc(sizeof(JsonbValue)); + replacevalue->type = jbvNull; + } + else + replacevalue = + JsonbToJsonbValue(DatumGetJsonbP(sbsrefstate->replacevalue)); + + /* + * In case if the input container is null, set up an empty jsonb and + * proceed with the assignment. + */ + if (*op->resnull) + { + JsonbValue *newSource = (JsonbValue *) palloc(sizeof(JsonbValue)); + + /* + * To avoid any surprising results, set up an empty jsonb array in case + * of an array is expected (i.e. the first subscript is integer), + * otherwise jsonb object. + */ + if (workspace->expectArray) + { + newSource->type = jbvArray; + newSource->val.array.nElems = 0; + newSource->val.array.rawScalar = false; + } + else + { + newSource->type = jbvObject; + newSource->val.object.nPairs = 0; + } + + jsonbSource = JsonbValueToJsonb(newSource); + *op->resnull = false; + } + else + jsonbSource = DatumGetJsonbP(*op->resvalue); + + *op->resvalue = jsonb_set_element(jsonbSource, + workspace->index, + sbsrefstate->numupper, + replacevalue); + /* The result is never NULL, so no need to change *op->resnull */ +} + +/* + * Compute old jsonb element value for a SubscriptingRef assignment + * expression. Will only be called if the new-value subexpression + * contains SubscriptingRef or FieldStore. This is the same as the + * regular fetch case, except that we have to handle a null jsonb, + * and the value should be stored into the SubscriptingRefState's + * prevvalue/prevnull fields. + */ +static void +jsonb_subscript_fetch_old(ExprState *state, + ExprEvalStep *op, + ExprContext *econtext) +{ + SubscriptingRefState *sbsrefstate = op->d.sbsref.state; + + if (*op->resnull) + { + /* whole jsonb is null, so any element is too */ + sbsrefstate->prevvalue = (Datum) 0; + sbsrefstate->prevnull = true; + } + else + { + Jsonb *jsonbSource = DatumGetJsonbP(*op->resvalue); + sbsrefstate->prevvalue = jsonb_get_element(jsonbSource, + sbsrefstate->upperindex, + sbsrefstate->numupper, + &sbsrefstate->prevnull, + false); + } +} + +/* + * Set up execution state for a jsonb subscript operation. Opposite to the + * arrays subscription, there is no limit for number of subscripts as jsonb + * type itself doesn't have nesting limits. + */ +static void +jsonb_exec_setup(const SubscriptingRef *sbsref, + SubscriptingRefState *sbsrefstate, + SubscriptExecSteps *methods) +{ + JsonbSubWorkspace *workspace; + ListCell *lc; + int nupper = sbsref->refupperindexpr->length; + char *ptr; + + /* Allocate type-specific workspace with space for per-subscript data */ + workspace = palloc0(MAXALIGN(sizeof(JsonbSubWorkspace)) + + nupper * (sizeof(Datum) + sizeof(Oid))); + workspace->expectArray = false; + ptr = ((char *) workspace) + MAXALIGN(sizeof(JsonbSubWorkspace)); + workspace->indexOid = (Oid *) ptr; + ptr += nupper * sizeof(Oid); + workspace->index = (Datum *) ptr; + + sbsrefstate->workspace = workspace; + + /* Collect subscript data types necessary at execution time */ + foreach(lc, sbsref->refupperindexpr) + { + Node *expr = lfirst(lc); + int i = foreach_current_index(lc); + + workspace->indexOid[i] = exprType(expr); + } + + /* + * Pass back pointers to appropriate step execution functions. + */ + methods->sbs_check_subscripts = jsonb_subscript_check_subscripts; + methods->sbs_fetch = jsonb_subscript_fetch; + methods->sbs_assign = jsonb_subscript_assign; + methods->sbs_fetch_old = jsonb_subscript_fetch_old; +} + +/* + * jsonb_subscript_handler + * Subscripting handler for jsonb. + * + */ +Datum +jsonb_subscript_handler(PG_FUNCTION_ARGS) +{ + static const SubscriptRoutines sbsroutines = { + .transform = jsonb_subscript_transform, + .exec_setup = jsonb_exec_setup, + .fetch_strict = true, /* fetch returns NULL for NULL inputs */ + .fetch_leakproof = true, /* fetch returns NULL for bad subscript */ + .store_leakproof = false /* ... but assignment throws error */ + }; + + PG_RETURN_POINTER(&sbsroutines); +} diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c index 69100feab7..5a0ba6b220 100644 --- a/src/backend/utils/adt/jsonfuncs.c +++ b/src/backend/utils/adt/jsonfuncs.c @@ -461,18 +461,18 @@ static Datum populate_domain(DomainIOData *io, Oid typid, const char *colname, /* functions supporting jsonb_delete, jsonb_set and jsonb_concat */ static JsonbValue *IteratorConcat(JsonbIterator **it1, JsonbIterator **it2, JsonbParseState **state); -static JsonbValue *setPath(JsonbIterator **it, Datum *path_elems, +extern JsonbValue *setPath(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, - JsonbParseState **st, int level, Jsonb *newval, + JsonbParseState **st, int level, JsonbValue *newval, int op_type); static void setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 npairs, int op_type); + JsonbValue *newval, uint32 npairs, int op_type); static void setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, - int level, Jsonb *newval, uint32 nelems, int op_type); -static void addJsonbToParseState(JsonbParseState **jbps, Jsonb *jb); + int level, + JsonbValue *newval, uint32 nelems, int op_type); /* function supporting iterate_json_values */ static void iterate_values_scalar(void *state, char *token, JsonTokenType tokentype); @@ -1448,13 +1448,9 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); Datum *pathtext; bool *pathnulls; + bool isnull; int npath; - int i; - bool have_object = false, - have_array = false; - JsonbValue *jbvp = NULL; - JsonbValue jbvbuf; - JsonbContainer *container; + Datum res; /* * If the array contains any null elements, return NULL, on the grounds @@ -1469,9 +1465,26 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) deconstruct_array(path, TEXTOID, -1, false, TYPALIGN_INT, &pathtext, &pathnulls, &npath); - /* Identify whether we have object, array, or scalar at top-level */ - container = &jb->root; + res = jsonb_get_element(jb, pathtext, npath, &isnull, as_text); + if (isnull) + PG_RETURN_NULL(); + else + PG_RETURN_DATUM(res); +} + +Datum +jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text) +{ + JsonbContainer *container = &jb->root; + JsonbValue *jbvp = NULL; + int i; + bool have_object = false, + have_array = false; + + *isnull = false; + + /* Identify whether we have object, array, or scalar at top-level */ if (JB_ROOT_IS_OBJECT(jb)) have_object = true; else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb)) @@ -1496,7 +1509,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) { if (as_text) { - PG_RETURN_TEXT_P(cstring_to_text(JsonbToCString(NULL, + return PointerGetDatum(cstring_to_text(JsonbToCString(NULL, container, VARSIZE(jb)))); } @@ -1512,22 +1525,25 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) if (have_object) { jbvp = getKeyJsonValueFromContainer(container, - VARDATA(pathtext[i]), - VARSIZE(pathtext[i]) - VARHDRSZ, - &jbvbuf); + VARDATA(path[i]), + VARSIZE(path[i]) - VARHDRSZ, + NULL); } else if (have_array) { long lindex; uint32 index; - char *indextext = TextDatumGetCString(pathtext[i]); + char *indextext = TextDatumGetCString(path[i]); char *endptr; errno = 0; lindex = strtol(indextext, &endptr, 10); if (endptr == indextext || *endptr != '\0' || errno != 0 || lindex > INT_MAX || lindex < INT_MIN) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } if (lindex >= 0) { @@ -1545,7 +1561,10 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) nelements = JsonContainerSize(container); if (-lindex > nelements) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } else index = nelements + lindex; } @@ -1555,11 +1574,15 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) else { /* scalar, extraction yields a null */ - PG_RETURN_NULL(); + *isnull = true; + return PointerGetDatum(NULL); } if (jbvp == NULL) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } else if (i == npath - 1) break; @@ -1581,9 +1604,12 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) if (as_text) { if (jbvp->type == jbvNull) - PG_RETURN_NULL(); + { + *isnull = true; + return PointerGetDatum(NULL); + } - PG_RETURN_TEXT_P(JsonbValueAsText(jbvp)); + return PointerGetDatum(JsonbValueAsText(jbvp)); } else { @@ -1594,6 +1620,28 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text) } } +Datum +jsonb_set_element(Jsonb* jb, Datum *path, int path_len, + JsonbValue *newval) +{ + JsonbValue *res; + JsonbParseState *state = NULL; + JsonbIterator *it; + bool *path_nulls = palloc0(path_len * sizeof(bool)); + + if (newval->type == jbvArray && newval->val.array.rawScalar) + *newval = newval->val.array.elems[0]; + + it = JsonbIteratorInit(&jb->root); + + res = setPath(&it, path, path_nulls, path_len, &state, 0, + newval, JB_PATH_CREATE); + + pfree(path_nulls); + + PG_RETURN_JSONB_P(JsonbValueToJsonb(res)); +} + /* * Return the text representation of the given JsonbValue. */ @@ -4151,58 +4199,6 @@ jsonb_strip_nulls(PG_FUNCTION_ARGS) PG_RETURN_POINTER(JsonbValueToJsonb(res)); } -/* - * Add values from the jsonb to the parse state. - * - * If the parse state container is an object, the jsonb is pushed as - * a value, not a key. - * - * This needs to be done using an iterator because pushJsonbValue doesn't - * like getting jbvBinary values, so we can't just push jb as a whole. - */ -static void -addJsonbToParseState(JsonbParseState **jbps, Jsonb *jb) -{ - JsonbIterator *it; - JsonbValue *o = &(*jbps)->contVal; - JsonbValue v; - JsonbIteratorToken type; - - it = JsonbIteratorInit(&jb->root); - - Assert(o->type == jbvArray || o->type == jbvObject); - - if (JB_ROOT_IS_SCALAR(jb)) - { - (void) JsonbIteratorNext(&it, &v, false); /* skip array header */ - Assert(v.type == jbvArray); - (void) JsonbIteratorNext(&it, &v, false); /* fetch scalar value */ - - switch (o->type) - { - case jbvArray: - (void) pushJsonbValue(jbps, WJB_ELEM, &v); - break; - case jbvObject: - (void) pushJsonbValue(jbps, WJB_VALUE, &v); - break; - default: - elog(ERROR, "unexpected parent of nested structure"); - } - } - else - { - while ((type = JsonbIteratorNext(&it, &v, false)) != WJB_DONE) - { - if (type == WJB_KEY || type == WJB_VALUE || type == WJB_ELEM) - (void) pushJsonbValue(jbps, type, &v); - else - (void) pushJsonbValue(jbps, type, NULL); - } - } - -} - /* * SQL function jsonb_pretty (jsonb) * @@ -4474,7 +4470,8 @@ jsonb_set(PG_FUNCTION_ARGS) { Jsonb *in = PG_GETARG_JSONB_P(0); ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); - Jsonb *newval = PG_GETARG_JSONB_P(2); + Jsonb *newjsonb = PG_GETARG_JSONB_P(2); + JsonbValue *newval = JsonbToJsonbValue(newjsonb); bool create = PG_GETARG_BOOL(3); JsonbValue *res = NULL; Datum *path_elems; @@ -4632,7 +4629,8 @@ jsonb_insert(PG_FUNCTION_ARGS) { Jsonb *in = PG_GETARG_JSONB_P(0); ArrayType *path = PG_GETARG_ARRAYTYPE_P(1); - Jsonb *newval = PG_GETARG_JSONB_P(2); + Jsonb *newjsonb = PG_GETARG_JSONB_P(2); + JsonbValue *newval = JsonbToJsonbValue(newjsonb); bool after = PG_GETARG_BOOL(3); JsonbValue *res = NULL; Datum *path_elems; @@ -4787,10 +4785,10 @@ IteratorConcat(JsonbIterator **it1, JsonbIterator **it2, * All path elements before the last must already exist * whatever bits in op_type are set, or nothing is done. */ -static JsonbValue * +JsonbValue * setPath(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, - JsonbParseState **st, int level, Jsonb *newval, int op_type) + JsonbParseState **st, int level, JsonbValue *newval, int op_type) { JsonbValue v; JsonbIteratorToken r; @@ -4843,11 +4841,11 @@ setPath(JsonbIterator **it, Datum *path_elems, static void setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 npairs, int op_type) + JsonbValue *newval, uint32 npairs, int op_type) { - JsonbValue v; int i; - JsonbValue k; + JsonbValue k, + v; bool done = false; if (level >= path_len || path_nulls[level]) @@ -4864,7 +4862,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, newkey.val.string.val = VARDATA_ANY(path_elems[level]); (void) pushJsonbValue(st, WJB_KEY, &newkey); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } for (i = 0; i < npairs; i++) @@ -4895,7 +4893,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, if (!(op_type & JB_PATH_DELETE)) { (void) pushJsonbValue(st, WJB_KEY, &k); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } done = true; } @@ -4918,7 +4916,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, newkey.val.string.val = VARDATA_ANY(path_elems[level]); (void) pushJsonbValue(st, WJB_KEY, &newkey); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_VALUE, newval); } (void) pushJsonbValue(st, r, &k); @@ -4950,7 +4948,7 @@ setPathObject(JsonbIterator **it, Datum *path_elems, bool *path_nulls, static void setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, int path_len, JsonbParseState **st, int level, - Jsonb *newval, uint32 nelems, int op_type) + JsonbValue *newval, uint32 nelems, int op_type) { JsonbValue v; int idx, @@ -4998,7 +4996,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, (op_type & JB_PATH_CREATE_OR_INSERT)) { Assert(newval != NULL); - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); done = true; } @@ -5014,7 +5012,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, r = JsonbIteratorNext(it, &v, true); /* skip */ if (op_type & (JB_PATH_INSERT_BEFORE | JB_PATH_CREATE)) - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); /* * We should keep current value only in case of @@ -5025,7 +5023,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, (void) pushJsonbValue(st, r, &v); if (op_type & (JB_PATH_INSERT_AFTER | JB_PATH_REPLACE)) - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); done = true; } @@ -5059,7 +5057,7 @@ setPathArray(JsonbIterator **it, Datum *path_elems, bool *path_nulls, if ((op_type & JB_PATH_CREATE_OR_INSERT) && !done && level == path_len - 1 && i == nelems - 1) { - addJsonbToParseState(st, newval); + (void) pushJsonbValue(st, WJB_ELEM, newval); } } } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 139f4a08bd..feae8cc4b0 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11239,6 +11239,10 @@ { oid => '9256', descr => 'raw array subscripting support', proname => 'raw_array_subscript_handler', prorettype => 'internal', proargtypes => 'internal', prosrc => 'raw_array_subscript_handler' }, +# type subscripting support +{ oid => '6098', descr => 'jsonb subscripting logic', + proname => 'jsonb_subscript_handler', prorettype => 'internal', + proargtypes => 'internal', prosrc => 'jsonb_subscript_handler' }, # collation management functions { oid => '3445', descr => 'import collations from operating system', diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index 62018f063a..4a530ca907 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -444,7 +444,8 @@ { oid => '3802', array_type_oid => '3807', descr => 'Binary JSON', typname => 'jsonb', typlen => '-1', typbyval => 'f', typcategory => 'U', typinput => 'jsonb_in', typoutput => 'jsonb_out', typreceive => 'jsonb_recv', - typsend => 'jsonb_send', typalign => 'i', typstorage => 'x' }, + typsend => 'jsonb_send', typalign => 'i', typstorage => 'x', + typsubscript => 'jsonb_subscript_handler' }, { oid => '4072', array_type_oid => '4073', descr => 'JSON path', typname => 'jsonpath', typlen => '-1', typbyval => 'f', typcategory => 'U', typinput => 'jsonpath_in', typoutput => 'jsonpath_out', diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h index 5860011693..64f1ccbf77 100644 --- a/src/include/utils/jsonb.h +++ b/src/include/utils/jsonb.h @@ -392,6 +392,7 @@ extern JsonbValue *pushJsonbValue(JsonbParseState **pstate, extern JsonbIterator *JsonbIteratorInit(JsonbContainer *container); extern JsonbIteratorToken JsonbIteratorNext(JsonbIterator **it, JsonbValue *val, bool skipNested); +extern JsonbValue *JsonbToJsonbValue(Jsonb *jsonb); extern Jsonb *JsonbValueToJsonb(JsonbValue *val); extern bool JsonbDeepContains(JsonbIterator **val, JsonbIterator **mContained); @@ -407,5 +408,8 @@ extern char *JsonbToCStringIndent(StringInfo out, JsonbContainer *in, extern bool JsonbExtractScalar(JsonbContainer *jbc, JsonbValue *res); extern const char *JsonbTypeName(JsonbValue *jb); - +extern Datum jsonb_set_element(Jsonb *jb, Datum *path, int path_len, + JsonbValue *newval); +extern Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath, + bool *isnull, bool as_text); #endif /* __JSONB_H__ */ diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out index 1e6c6ef200..bb3f25ec3f 100644 --- a/src/test/regress/expected/jsonb.out +++ b/src/test/regress/expected/jsonb.out @@ -4599,7 +4599,7 @@ select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'use_ {"a": 1, "b": null} (1 row) -\pset null +\pset null '' -- jsonb_insert select jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"'); jsonb_insert @@ -4729,6 +4729,276 @@ HINT: Try using the function jsonb_set to replace key value. select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"', true); ERROR: cannot replace existing key HINT: Try using the function jsonb_set to replace key value. +-- jsonb subscript +select ('123'::jsonb)['a']; + jsonb +------- + +(1 row) + +select ('123'::jsonb)[0]; + jsonb +------- + +(1 row) + +select ('123'::jsonb)[NULL]; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)['a']; + jsonb +------- + 1 +(1 row) + +select ('{"a": 1}'::jsonb)[0]; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)['not_exist']; + jsonb +------- + +(1 row) + +select ('{"a": 1}'::jsonb)[NULL]; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)['a']; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[0]; + jsonb +------- + 1 +(1 row) + +select ('[1, "2", null]'::jsonb)['1']; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1.0]; +ERROR: subscript type is not supported +LINE 1: select ('[1, "2", null]'::jsonb)[1.0]; + ^ +HINT: Jsonb subscript must be coercet to either integer or text +select ('[1, "2", null]'::jsonb)[2]; + jsonb +------- + null +(1 row) + +select ('[1, "2", null]'::jsonb)[3]; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[-2]; + jsonb +------- + "2" +(1 row) + +select ('[1, "2", null]'::jsonb)[1]['a']; + jsonb +------- + +(1 row) + +select ('[1, "2", null]'::jsonb)[1][0]; + jsonb +------- + +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['b']; + jsonb +------- + "c" +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']; + jsonb +----------- + [1, 2, 3] +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d'][1]; + jsonb +------- + 2 +(1 row) + +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']['a']; + jsonb +------- + +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']; + jsonb +--------------- + {"a2": "aaa"} +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']; + jsonb +------- + "aaa" +(1 row) + +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']['a3']; + jsonb +------- + +(1 row) + +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1']; + jsonb +----------------------- + ["aaa", "bbb", "ccc"] +(1 row) + +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1'][2]; + jsonb +------- + "ccc" +(1 row) + +-- slices are not supported +select ('{"a": 1}'::jsonb)['a':'b']; +ERROR: jsonb subscript does not support slices +LINE 1: select ('{"a": 1}'::jsonb)['a':'b']; + ^ +select ('[1, "2", null]'::jsonb)[1:2]; +ERROR: jsonb subscript does not support slices +LINE 1: select ('[1, "2", null]'::jsonb)[1:2]; + ^ +select ('[1, "2", null]'::jsonb)[:2]; +ERROR: jsonb subscript does not support slices +LINE 1: select ('[1, "2", null]'::jsonb)[:2]; + ^ +select ('[1, "2", null]'::jsonb)[1:]; +ERROR: jsonb subscript does not support slices +LINE 1: select ('[1, "2", null]'::jsonb)[1:]; + ^ +select ('[1, "2", null]'::jsonb)[:]; +ERROR: jsonb subscript does not support slices +create TEMP TABLE test_jsonb_subscript ( + id int, + test_json jsonb +); +insert into test_jsonb_subscript values +(1, '{}'), -- empty jsonb +(2, '{"key": "value"}'); -- jsonb with data +-- update empty jsonb +update test_jsonb_subscript set test_json['a'] = '1' where id = 1; +select * from test_jsonb_subscript; + id | test_json +----+------------------ + 2 | {"key": "value"} + 1 | {"a": 1} +(2 rows) + +-- update jsonb with some data +update test_jsonb_subscript set test_json['a'] = '1' where id = 2; +select * from test_jsonb_subscript; + id | test_json +----+-------------------------- + 1 | {"a": 1} + 2 | {"a": 1, "key": "value"} +(2 rows) + +-- replace jsonb +update test_jsonb_subscript set test_json['a'] = '"test"'; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------- + 1 | {"a": "test"} + 2 | {"a": "test", "key": "value"} +(2 rows) + +-- replace by object +update test_jsonb_subscript set test_json['a'] = '{"b": 1}'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------- + 1 | {"a": {"b": 1}} + 2 | {"a": {"b": 1}, "key": "value"} +(2 rows) + +-- replace by array +update test_jsonb_subscript set test_json['a'] = '[1, 2, 3]'::jsonb; +select * from test_jsonb_subscript; + id | test_json +----+---------------------------------- + 1 | {"a": [1, 2, 3]} + 2 | {"a": [1, 2, 3], "key": "value"} +(2 rows) + +-- use jsonb subscription in where clause +select * from test_jsonb_subscript where test_json['key'] = '"value"'; + id | test_json +----+---------------------------------- + 2 | {"a": [1, 2, 3], "key": "value"} +(1 row) + +select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; + id | test_json +----+----------- +(0 rows) + +select * from test_jsonb_subscript where test_json['key'] = '"wrong_value"'; + id | test_json +----+----------- +(0 rows) + +-- NULL +update test_jsonb_subscript set test_json[NULL] = '1'; +ERROR: jsonb subscript in assignment must not be null +update test_jsonb_subscript set test_json['another_key'] = NULL; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------------------- + 1 | {"a": [1, 2, 3], "another_key": null} + 2 | {"a": [1, 2, 3], "key": "value", "another_key": null} +(2 rows) + +-- NULL as jsonb source +insert into test_jsonb_subscript values (3, NULL); +update test_jsonb_subscript set test_json['a'] = '1' where id = 3; +select * from test_jsonb_subscript; + id | test_json +----+------------------------------------------------------- + 1 | {"a": [1, 2, 3], "another_key": null} + 2 | {"a": [1, 2, 3], "key": "value", "another_key": null} + 3 | {"a": 1} +(3 rows) + +update test_jsonb_subscript set test_json = NULL where id = 3; +update test_jsonb_subscript set test_json[0] = '1'; +select * from test_jsonb_subscript; + id | test_json +----+--------------------------------------------------------------- + 1 | {"0": 1, "a": [1, 2, 3], "another_key": null} + 2 | {"0": 1, "a": [1, 2, 3], "key": "value", "another_key": null} + 3 | [1] +(3 rows) + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); to_tsvector diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql index b6409767f6..20aa8fe0e2 100644 --- a/src/test/regress/sql/jsonb.sql +++ b/src/test/regress/sql/jsonb.sql @@ -1177,7 +1177,7 @@ select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'retu select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'delete_key') as delete_key; select jsonb_set_lax('{"a":1,"b":2}', '{b}', null, null_value_treatment => 'use_json_null') as use_json_null; -\pset null +\pset null '' -- jsonb_insert select jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"'); @@ -1208,6 +1208,88 @@ select jsonb_insert('{"a": {"b": "value"}}', '{a, c}', '"new_value"', true); select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"'); select jsonb_insert('{"a": {"b": "value"}}', '{a, b}', '"new_value"', true); +-- jsonb subscript +select ('123'::jsonb)['a']; +select ('123'::jsonb)[0]; +select ('123'::jsonb)[NULL]; +select ('{"a": 1}'::jsonb)['a']; +select ('{"a": 1}'::jsonb)[0]; +select ('{"a": 1}'::jsonb)['not_exist']; +select ('{"a": 1}'::jsonb)[NULL]; +select ('[1, "2", null]'::jsonb)['a']; +select ('[1, "2", null]'::jsonb)[0]; +select ('[1, "2", null]'::jsonb)['1']; +select ('[1, "2", null]'::jsonb)[1.0]; +select ('[1, "2", null]'::jsonb)[2]; +select ('[1, "2", null]'::jsonb)[3]; +select ('[1, "2", null]'::jsonb)[-2]; +select ('[1, "2", null]'::jsonb)[1]['a']; +select ('[1, "2", null]'::jsonb)[1][0]; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['b']; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d'][1]; +select ('{"a": 1, "b": "c", "d": [1, 2, 3]}'::jsonb)['d']['a']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']; +select ('{"a": {"a1": {"a2": "aaa"}}, "b": "bbb", "c": "ccc"}'::jsonb)['a']['a1']['a2']['a3']; +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1']; +select ('{"a": ["a1", {"b1": ["aaa", "bbb", "ccc"]}], "b": "bb"}'::jsonb)['a'][1]['b1'][2]; + +-- slices are not supported +select ('{"a": 1}'::jsonb)['a':'b']; +select ('[1, "2", null]'::jsonb)[1:2]; +select ('[1, "2", null]'::jsonb)[:2]; +select ('[1, "2", null]'::jsonb)[1:]; +select ('[1, "2", null]'::jsonb)[:]; + +create TEMP TABLE test_jsonb_subscript ( + id int, + test_json jsonb +); + +insert into test_jsonb_subscript values +(1, '{}'), -- empty jsonb +(2, '{"key": "value"}'); -- jsonb with data + +-- update empty jsonb +update test_jsonb_subscript set test_json['a'] = '1' where id = 1; +select * from test_jsonb_subscript; + +-- update jsonb with some data +update test_jsonb_subscript set test_json['a'] = '1' where id = 2; +select * from test_jsonb_subscript; + +-- replace jsonb +update test_jsonb_subscript set test_json['a'] = '"test"'; +select * from test_jsonb_subscript; + +-- replace by object +update test_jsonb_subscript set test_json['a'] = '{"b": 1}'::jsonb; +select * from test_jsonb_subscript; + +-- replace by array +update test_jsonb_subscript set test_json['a'] = '[1, 2, 3]'::jsonb; +select * from test_jsonb_subscript; + +-- use jsonb subscription in where clause +select * from test_jsonb_subscript where test_json['key'] = '"value"'; +select * from test_jsonb_subscript where test_json['key_doesnt_exists'] = '"value"'; +select * from test_jsonb_subscript where test_json['key'] = '"wrong_value"'; + +-- NULL +update test_jsonb_subscript set test_json[NULL] = '1'; +update test_jsonb_subscript set test_json['another_key'] = NULL; +select * from test_jsonb_subscript; + +-- NULL as jsonb source +insert into test_jsonb_subscript values (3, NULL); +update test_jsonb_subscript set test_json['a'] = '1' where id = 3; +select * from test_jsonb_subscript; + +update test_jsonb_subscript set test_json = NULL where id = 3; +update test_jsonb_subscript set test_json[0] = '1'; +select * from test_jsonb_subscript; + -- jsonb to tsvector select to_tsvector('{"a": "aaa bbb ddd ccc", "b": ["eee fff ggg"], "c": {"d": "hhh iii"}}'::jsonb); -- 2.21.0 --6snw7vcvuujiswgu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v43-0002-Filling-gaps-in-jsonb.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* [PATCH v20260912 4/4] Make printtup a bit faster (intermediate state). @ 2024-09-12 10:03 Andy Fan <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Andy Fan @ 2024-09-12 10:03 UTC (permalink / raw) Currently the out function usually allocate its own memory and fill it with the cstring. After the printtup get the cstring, printtup computes it string length and copy it to its own StringInfo. So there are some wastage in this workflow. In the desired case, out function should take a StringInfo as a input and fill the data to StringInfo's buffer directly. Within this way, there is no extra memory allocate, memory copy and probably avoid the most strlen since the most of the outfunction can compute it easily. for example a). snprintf return the length encoded string, b). the varlena's header has a strlen. c). we know the start position before we encode a Datum and we know the end position after the Datum encoding, so the length would be similar as 'end_pos - start_pos'. Since we have 79 out functions to change, this patch just finish part of them by using a new print function and wish a review of it. If there are anything wrong, it is better know them earlier. --- src/backend/access/common/printtup.c | 80 ++++++++++++++++++-- src/backend/utils/adt/char.c | 32 ++++++++ src/backend/utils/adt/date.c | 74 ++++++++++++++++++- src/backend/utils/adt/datetime.c | 17 ++++- src/backend/utils/adt/float.c | 53 +++++++++++++- src/backend/utils/adt/int.c | 32 ++++++++ src/backend/utils/adt/int8.c | 16 ++++ src/backend/utils/adt/numeric.c | 68 +++++++++++++++-- src/backend/utils/adt/oid.c | 16 ++++ src/backend/utils/adt/timestamp.c | 106 ++++++++++++++++++++++++++- src/backend/utils/adt/varchar.c | 25 +++++++ src/backend/utils/adt/varlena.c | 16 ++++ src/include/catalog/pg_proc.dat | 83 ++++++++++++++++++++- src/include/lib/stringinfo.h | 19 +++++ src/include/utils/date.h | 2 +- src/include/utils/datetime.h | 8 +- 16 files changed, 618 insertions(+), 29 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 616bdafd395..860e67cfcc9 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -19,6 +19,7 @@ #include "libpq/pqformat.h" #include "libpq/protocol.h" #include "tcop/pquery.h" +#include "utils/fmgroids.h" #include "utils/lsyscache.h" #include "utils/memdebug.h" #include "utils/memutils.h" @@ -50,6 +51,7 @@ typedef struct bool typisvarlena; /* is it varlena (ie possibly toastable)? */ int16 format; /* format code for this column */ FmgrInfo finfo; /* Precomputed call info for output fn */ + FmgrInfo p_finfo; /* Precomputed call info for print fn if any */ } PrinttupAttrInfo; typedef struct @@ -244,6 +246,47 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, pq_endmessage_reuse(buf); } +static Oid +get_type_printfn_tmp(Oid type) +{ + switch(type) + { + case OIDOID: + return F_OIDPRINT; + case TEXTOID: + return F_TEXTPRINT; + case FLOAT4OID: + return F_FLOAT4PRINT; + case FLOAT8OID: + return F_FLOAT8PRINT; + case INT2OID: + return F_INT2PRINT; + case INT4OID: + return F_INT4PRINT; + case INT8OID: + return F_INT8PRINT; + case TIMEOID: + return F_TIMEPRINT; + case TIMETZOID: + return F_TIMETZPRINT; + case TIMESTAMPOID: + return F_TIMESTAMPPRINT; + case TIMESTAMPTZOID: + return F_TIMESTAMPTZPRINT; + case INTERVALOID: + return F_INTERVAL_PRINT; + case NUMERICOID: + return F_NUMERIC_PRINT; + case BPCHAROID: + return F_BPCHARPRINT; + case VARCHAROID: + return F_VARCHARPRINT; + case CHAROID: + return F_CHARPRINT; + } + return InvalidOid; +} + /* * Get the lookup info that printtup() needs */ @@ -275,10 +318,18 @@ printtup_prepare_info(DR_printtup *myState, TupleDesc typeinfo, int numAttrs) thisState->format = format; if (format == 0) { - getTypeOutputInfo(attr->atttypid, - &thisState->typoutput, - &thisState->typisvarlena); - fmgr_info(thisState->typoutput, &thisState->finfo); + Oid print_fn = get_type_printfn_tmp(attr->atttypid); + if (print_fn != InvalidOid) + fmgr_info(print_fn, &thisState->p_finfo); + else + { + getTypeOutputInfo(attr->atttypid, + &thisState->typoutput, + &thisState->typisvarlena); + fmgr_info(thisState->typoutput, &thisState->finfo); + /* mark print function is invalid */ + thisState->p_finfo.fn_oid = InvalidOid; + } } else if (format == 1) { @@ -356,10 +407,23 @@ printtup(TupleTableSlot *slot, DestReceiver *self) if (thisState->format == 0) { /* Text output */ - char *outputstr; - - outputstr = OutputFunctionCall(&thisState->finfo, attr); - pq_sendcountedtext(buf, outputstr, strlen(outputstr)); + if (thisState->p_finfo.fn_oid) + { + /* + * Use print function if it is defined. + * + * XXX: we can remove this if statement once we refactor all + * the out function. + */ + FunctionCall2(&thisState->p_finfo, attr, PointerGetDatum(buf)); + } + else + { + char *outputstr; + + outputstr = OutputFunctionCall(&thisState->finfo, attr); + pq_sendcountedtext(buf, outputstr, strlen(outputstr)); + } } else { diff --git a/src/backend/utils/adt/char.c b/src/backend/utils/adt/char.c index 698863924ee..6d1d9403c2d 100644 --- a/src/backend/utils/adt/char.c +++ b/src/backend/utils/adt/char.c @@ -83,6 +83,38 @@ charout(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +charprint(PG_FUNCTION_ARGS) +{ + char ch = PG_GETARG_CHAR(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + char *result; + uint32 data_len; + + result = outStringReserveLen(buf, 5); + + if (IS_HIGHBIT_SET(ch)) + { + result[0] = '\\'; + result[1] = TOOCTAL(((unsigned char) ch) >> 6); + result[2] = TOOCTAL((((unsigned char) ch) >> 3) & 07); + result[3] = TOOCTAL(((unsigned char) ch) & 07); + result[4] = '\0'; + data_len = 4; + } + else + { + /* This produces acceptable results for 0x00 as well */ + result[0] = ch; + result[1] = '\0'; + data_len = 1; + } + + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * charrecv - converts external binary format to char * diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index c3327440380..4d606e7888f 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -196,6 +196,31 @@ date_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +date_print(PG_FUNCTION_ARGS) +{ + DateADT date = PG_GETARG_DATEADT(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + char *data; + uint32 data_len; + + struct pg_tm tt, + *tm = &tt; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + + if (DATE_NOT_FINITE(date)) + data_len = EncodeSpecialDate(date, data); + else + { + j2date(date + POSTGRES_EPOCH_JDATE, + &(tm->tm_year), &(tm->tm_mon), &(tm->tm_mday)); + data_len = EncodeDateOnly(tm, DateStyle, data); + } + outStringCompletePhase(buf, data_len); + PG_RETURN_VOID(); +} + /* * date_recv - converts external binary format to date */ @@ -291,13 +316,21 @@ make_date(PG_FUNCTION_ARGS) /* * Convert reserved date values to string. */ -void +int EncodeSpecialDate(DateADT dt, char *str) { if (DATE_IS_NOBEGIN(dt)) + { strcpy(str, EARLY); + /* the return value can be computed at compiling time. */ + return strlen(EARLY); + } else if (DATE_IS_NOEND(dt)) + { strcpy(str, LATE); + /* the return value can be computed at compiling time. */ + return strlen(LATE); + } else /* shouldn't happen */ elog(ERROR, "invalid argument for EncodeSpecialDate"); } @@ -1603,6 +1636,25 @@ time_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +time_print(PG_FUNCTION_ARGS) +{ + TimeADT time = PG_GETARG_TIMEADT(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + struct pg_tm tt, + *tm = &tt; + fsec_t fsec; + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + time2tm(time, tm, &fsec); + data_len = EncodeTimeOnly(tm, fsec, false, 0, DateStyle, data); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * time_recv - converts external binary format to time */ @@ -2417,6 +2469,26 @@ timetz_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +timetz_print(PG_FUNCTION_ARGS) +{ + TimeTzADT *time = PG_GETARG_TIMETZADT_P(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + struct pg_tm tt, + *tm = &tt; + fsec_t fsec; + char *data; + uint32 data_len; + int tz; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + timetz2tm(time, tm, &fsec, &tz); + data_len = EncodeTimeOnly(tm, fsec, true, tz, DateStyle, data); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * timetz_recv - converts external binary format to timetz */ diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c index 83c3c85305b..85da5780870 100644 --- a/src/backend/utils/adt/datetime.c +++ b/src/backend/utils/adt/datetime.c @@ -4346,9 +4346,10 @@ EncodeTimezone(char *str, int tz, int style) /* EncodeDateOnly() * Encode date as local time. */ -void +int EncodeDateOnly(struct pg_tm *tm, int style, char *str) { + char *start = str; Assert(tm->tm_mon >= 1 && tm->tm_mon <= MONTHS_PER_YEAR); switch (style) @@ -4420,6 +4421,7 @@ EncodeDateOnly(struct pg_tm *tm, int style, char *str) str += 3; } *str = '\0'; + return str - start; } @@ -4430,10 +4432,13 @@ EncodeDateOnly(struct pg_tm *tm, int style, char *str) * a time zone (the difference between time and timetz types), tz is the * numeric time zone offset, style is the date style, str is where to write the * output. + * + * returns the strlen of the encoded format. */ -void +int EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, char *str) { + char *start = str; str = pg_ultostr_zeropad(str, tm->tm_hour, 2); *str++ = ':'; str = pg_ultostr_zeropad(str, tm->tm_min, 2); @@ -4442,6 +4447,7 @@ EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, if (print_tz) str = EncodeTimezone(str, tz, style); *str = '\0'; + return str - start; } @@ -4460,11 +4466,14 @@ EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, * ISO - yyyy-mm-dd hh:mm:ss+/-tz * German - dd.mm.yyyy hh:mm:ss tz * XSD - yyyy-mm-ddThh:mm:ss.ss+/-tz + * + * return the strlen of the encoded data. */ -void +int EncodeDateTime(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, const char *tzn, int style, char *str) { int day; + char *start = str; Assert(tm->tm_mon >= 1 && tm->tm_mon <= MONTHS_PER_YEAR); @@ -4624,6 +4633,8 @@ EncodeDateTime(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, const char str += 3; } *str = '\0'; + + return str - start; } diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 13ceade129e..76ad00c60d9 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -373,6 +373,32 @@ float4out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(ascii); } + +Datum +float4print(PG_FUNCTION_ARGS) +{ + float4 num = PG_GETARG_FLOAT4(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + int data_len; + char *ascii; + int ndig = FLT_DIG + extra_float_digits; + + ascii = outStringReserveLen(buf, 32); + + if (extra_float_digits > 0) + data_len = float_to_shortest_decimal_buf(num, ascii); + else + data_len = pg_strfromd(ascii, 32, ndig, num); + if (data_len == -1) + { + /* XXX, think more of this. */ + elog(ERROR, "failed on float4print"); + } + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * float4recv - converts external binary format to float4 */ @@ -563,10 +589,35 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + int len; - PG_RETURN_CSTRING(float8out_internal(num)); + PG_RETURN_CSTRING(float8out_internal(num, NULL, &len)); } +Datum +float8print(PG_FUNCTION_ARGS) +{ + float8 num = PG_GETARG_FLOAT8(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + int data_len; + char *ascii; + + ascii = outStringReserveLen(buf, 32); + + float8out_internal(num, ascii, &data_len); + + if (data_len == -1) + { + /* XXX, think more of this. */ + elog(ERROR, "failed on float8print"); + } + + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + + /* * float8out_internal - guts of float8out() * diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index 4c894a49d5d..5a121a46b94 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -80,6 +80,22 @@ int2out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } + +Datum +int2print(PG_FUNCTION_ARGS) +{ + int16 arg1 = PG_GETARG_INT16(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, 7); + data_len = pg_itoa(arg1, data); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * int2recv - converts external binary format to int2 */ @@ -333,6 +349,22 @@ int4out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +int4print(PG_FUNCTION_ARGS) +{ + int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, 12); + data_len = pg_ltoa(arg1, data); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + + /* * int4recv - converts external binary format to int4 */ diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 19bb30f2d0f..8580c273792 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -76,6 +76,22 @@ int8out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +int8print(PG_FUNCTION_ARGS) +{ + int64 arg1 = PG_GETARG_INT64(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, MAXINT8LEN + 1); + data_len = pg_lltoa(arg1, data); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + + /* * int8recv - converts external binary format to int8 */ diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..84719a79ae8 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -509,7 +509,7 @@ static bool set_var_from_non_decimal_integer_str(const char *str, static void set_var_from_num(Numeric num, NumericVar *dest); static void init_var_from_num(Numeric num, NumericVar *dest); static void set_var_from_var(const NumericVar *value, NumericVar *dest); -static char *get_str_from_var(const NumericVar *var); +static char *get_str_from_var(const NumericVar *var, StringInfo buf); static char *get_str_from_var_sci(const NumericVar *var, int rscale); static void numericvar_serialize(StringInfo buf, const NumericVar *var); @@ -820,11 +820,52 @@ numeric_out(PG_FUNCTION_ARGS) */ init_var_from_num(num, &x); - str = get_str_from_var(&x); + str = get_str_from_var(&x, NULL); PG_RETURN_CSTRING(str); } +Datum +numeric_print(PG_FUNCTION_ARGS) +{ + Numeric num = PG_GETARG_NUMERIC(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + + NumericVar x; + + /* + * Handle NaN and infinities + */ + if (NUMERIC_IS_SPECIAL(num)) + { + const char* special_str; + char *data; + uint32 data_len; + + if (NUMERIC_IS_PINF(num)) + special_str = "Infinity"; + else if (NUMERIC_IS_NINF(num)) + special_str = "-Infinity"; + else + special_str = "NaN"; + + data_len = strlen(special_str) + 1; + data = outStringReserveLen(buf, data_len); + memcpy(data, special_str, data_len); + outStringCompletePhase(buf, data_len); + PG_RETURN_VOID(); + } + + /* + * Get the number in the variable format. + */ + init_var_from_num(num, &x); + + (void) get_str_from_var(&x, buf); + + PG_RETURN_VOID(); +} + /* * numeric_is_nan() - * @@ -1027,7 +1068,7 @@ numeric_normalize(Numeric num) init_var_from_num(num, &x); - str = get_str_from_var(&x); + str = get_str_from_var(&x, NULL); /* If there's no decimal point, there's certainly nothing to remove. */ if (strchr(str, '.') != NULL) @@ -7251,7 +7292,7 @@ set_var_from_var(const NumericVar *value, NumericVar *dest) * Returns a palloc'd string. */ static char * -get_str_from_var(const NumericVar *var) +get_str_from_var(const NumericVar *var, StringInfo buf) { int dscale; char *str; @@ -7279,7 +7320,14 @@ get_str_from_var(const NumericVar *var) if (i <= 0) i = 1; - str = palloc(i + dscale + DEC_DIGITS + 2); + if (buf == NULL) + { + str = palloc(i + dscale + DEC_DIGITS + 2); + } + else + { + str = outStringReserveLen(buf, i + dscale + DEC_DIGITS + 2); + } cp = str; /* @@ -7378,6 +7426,12 @@ get_str_from_var(const NumericVar *var) * terminate the string and return it */ *cp = '\0'; + + if (buf != NULL) + { + uint32 data_len = cp - str; + outStringCompletePhase(buf, data_len); + } return str; } @@ -7451,7 +7505,7 @@ get_str_from_var_sci(const NumericVar *var, int rscale) power_ten_int(exponent, &tmp_var); div_var(var, &tmp_var, &tmp_var, rscale, true, true); - sig_out = get_str_from_var(&tmp_var); + sig_out = get_str_from_var(&tmp_var, NULL); free_var(&tmp_var); @@ -8004,7 +8058,7 @@ numericvar_to_double_no_overflow(const NumericVar *var) double val; char *endptr; - tmp = get_str_from_var(var); + tmp = get_str_from_var(var, NULL); /* unlike float8in, we ignore ERANGE from strtod */ val = strtod(tmp, &endptr); diff --git a/src/backend/utils/adt/oid.c b/src/backend/utils/adt/oid.c index a3419728971..96df114eaf6 100644 --- a/src/backend/utils/adt/oid.c +++ b/src/backend/utils/adt/oid.c @@ -53,6 +53,22 @@ oidout(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +oidprint(PG_FUNCTION_ARGS) +{ + Oid o = PG_GETARG_OID(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + uint32 data_len; + char *data; + + /* 12 is the max length for an oid's text presentation. */ + data = outStringReserveLen(buf, 12); + data_len = pg_snprintf(data, 12, "%u", o); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * oidrecv - converts external binary format to oid */ diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index 288d696be77..27b546deb74 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -87,7 +87,7 @@ static bool AdjustIntervalForTypmod(Interval *interval, int32 typmod, static TimestampTz timestamp2timestamptz(Timestamp timestamp); static Timestamp timestamptz2timestamp(TimestampTz timestamp); -static void EncodeSpecialInterval(const Interval *interval, char *str); +static int EncodeSpecialInterval(const Interval *interval, char *str); static void interval_um_internal(const Interval *interval, Interval *result); /* common code for timestamptypmodin and timestamptztypmodin */ @@ -244,6 +244,33 @@ timestamp_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +timestamp_print(PG_FUNCTION_ARGS) +{ + Timestamp timestamp = PG_GETARG_TIMESTAMP(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + struct pg_tm tt, + *tm = &tt; + fsec_t fsec; + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + + if (TIMESTAMP_NOT_FINITE(timestamp)) + data_len = EncodeSpecialTimestamp(timestamp, data); + else if (timestamp2tm(timestamp, NULL, tm, &fsec, NULL, NULL) == 0) + data_len = EncodeDateTime(tm, fsec, false, 0, NULL, DateStyle, data); + else + ereport(ERROR, + (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), + errmsg("timestamp out of range"))); + + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * timestamp_recv - converts external binary format to timestamp */ @@ -789,6 +816,36 @@ timestamptz_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +timestamptz_print(PG_FUNCTION_ARGS) +{ + TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + int tz; + const char *tzn; + struct pg_tm tt, + *tm = &tt; + fsec_t fsec; + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + + if (TIMESTAMP_NOT_FINITE(timestamp)) + data_len = EncodeSpecialTimestamp(timestamp, data); + else if (timestamp2tm(timestamp, &tz, tm, &fsec, &tzn, NULL) == 0) + data_len = EncodeDateTime(tm, fsec, true, tz, tzn, DateStyle, data); + else + ereport(ERROR, + (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), + errmsg("timestamp out of range"))); + + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + + /* * timestamptz_recv - converts external binary format to timestamptz */ @@ -983,6 +1040,35 @@ interval_out(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(result); } +Datum +interval_print(PG_FUNCTION_ARGS) +{ + Interval *span = PG_GETARG_INTERVAL_P(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + struct pg_itm tt, + *itm = &tt; + char *data; + uint32 data_len; + + data = outStringReserveLen(buf, MAXDATELEN + 1); + + if (INTERVAL_NOT_FINITE(span)) + data_len = EncodeSpecialInterval(span, data); + else + { + interval2itm(*span, itm); + EncodeInterval(itm, IntervalStyle, data); + /* + * XXX: making EncodeInterval returns a string len is error-prone for me. + * so call strlen directly on the result. + */ + data_len = strlen(data); + } + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + /* * interval_recv - converts external binary format to interval */ @@ -1577,26 +1663,40 @@ out_of_range: /* EncodeSpecialTimestamp() * Convert reserved timestamp data type to string. */ -void +int EncodeSpecialTimestamp(Timestamp dt, char *str) { if (TIMESTAMP_IS_NOBEGIN(dt)) + { strcpy(str, EARLY); + return strlen(EARLY); + } else if (TIMESTAMP_IS_NOEND(dt)) + { strcpy(str, LATE); + return strlen(LATE); + } else /* shouldn't happen */ elog(ERROR, "invalid argument for EncodeSpecialTimestamp"); } -static void +static int EncodeSpecialInterval(const Interval *interval, char *str) { if (INTERVAL_IS_NOBEGIN(interval)) + { strcpy(str, EARLY); + return strlen(EARLY); + } else if (INTERVAL_IS_NOEND(interval)) + { strcpy(str, LATE); + return strlen(LATE); + } else /* shouldn't happen */ elog(ERROR, "invalid argument for EncodeSpecialInterval"); + + return 0; } Datum diff --git a/src/backend/utils/adt/varchar.c b/src/backend/utils/adt/varchar.c index a62e55eec19..e14c66999a8 100644 --- a/src/backend/utils/adt/varchar.c +++ b/src/backend/utils/adt/varchar.c @@ -223,6 +223,25 @@ bpcharout(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(TextDatumGetCString(txt)); } +Datum +bpcharprint(PG_FUNCTION_ARGS) +{ + Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + + /* XXX: improve here since we can put the cstring into buf directly. */ + char *data = TextDatumGetCString(txt); + uint32 data_len = strlen(data); + char *target; + + target = outStringReserveLen(buf, data_len); + memcpy(target, data, data_len); + outStringCompletePhase(buf, data_len); + + PG_RETURN_VOID(); +} + + /* * bpcharrecv - converts external binary format to bpchar */ @@ -520,6 +539,12 @@ varcharout(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(TextDatumGetCString(txt)); } +Datum +varcharprint(PG_FUNCTION_ARGS) +{ + return bpcharprint(fcinfo); +} + /* * varcharrecv - converts external binary format to varchar */ diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index c0ff51bd2fc..3fa6bff1182 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -293,6 +293,22 @@ textout(PG_FUNCTION_ARGS) PG_RETURN_CSTRING(TextDatumGetCString(txt)); } + +Datum +textprint(PG_FUNCTION_ARGS) +{ + text *txt = (text *) pg_detoast_datum((struct varlena *)PG_GETARG_POINTER(0)); + StringInfo buf = (StringInfo) PG_GETARG_POINTER(1); + uint32 text_len = VARSIZE(txt) - VARHDRSZ; + char *data; + + data = outStringReserveLen(buf, text_len); + memcpy(data, VARDATA(txt), text_len); + outStringCompletePhase(buf, text_len); + + PG_RETURN_VOID(); +} + /* * textrecv - converts external binary format to text */ diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fa9ae79082b..938f5e2d585 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -4877,7 +4877,6 @@ { oid => '1799', descr => 'I/O', proname => 'oidout', prorettype => 'cstring', proargtypes => 'oid', prosrc => 'oidout' }, - { oid => '3058', descr => 'concatenate values', proname => 'concat', provariadic => 'any', proisstrict => 'f', provolatile => 's', prorettype => 'text', proargtypes => 'any', @@ -12769,4 +12768,86 @@ proname => 'hashoid8extended', prorettype => 'int8', proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' }, +{ + oid => '9771', descr => 'I/O', + proname => 'oidprint', prorettype => 'void', proargtypes => 'oid internal', + prosrc => 'oidprint'}, +{ + oid => '8907', descr => 'I/O', + proname => 'textprint', prorettype => 'void', proargtypes => 'text internal', + prosrc => 'textprint' }, + +{ + oid => '9234', descr => 'I/O', + proname => 'float4print', prorettype => 'void', proargtypes => 'float4 internal', + prosrc => 'float4print' }, +{ + oid => '6313', descr => 'I/O', + proname => 'float8print', prorettype => 'void', proargtypes => 'float8 internal', + prosrc => 'float8print' }, + +{ + oid => '4099', descr => 'I/O', + proname => 'int2print', prorettype => 'void', proargtypes => 'int2 internal', + prosrc => 'int2print' }, +{ + oid => '4100', descr => 'I/O', + proname => 'int4print', prorettype => 'void', proargtypes => 'int4 internal', + prosrc => 'int4print' }, +{ + oid => '4551', descr => 'I/O', + proname => 'int8print', prorettype => 'void', proargtypes => 'int8 internal', + prosrc => 'int8print' }, + +{ + oid => '4552', descr => 'I/O', + proname => 'timeprint', prorettype => 'void', proargtypes => 'time internal', + prosrc => 'time_print' }, + +{ + oid => '4553', descr => 'I/O', + proname => 'timetzprint', prorettype => 'void', proargtypes => 'timetz internal', + prosrc => 'timetz_print' }, + +{ + oid => '4554', descr => 'I/O', + proname => 'dateprint', prorettype => 'void', proargtypes => 'date internal', + prosrc => 'date_print'}, + + +{ + oid => '4555', descr => 'I/O', + proname => 'timestampprint', prorettype => 'void', proargtypes => 'timestamp internal', + prosrc => 'timestamp_print'}, + +{ + oid => '4556', descr => 'I/O', + proname => 'timestamptzprint', prorettype => 'void', proargtypes => 'timestamptz internal', + prosrc => 'timestamptz_print'}, + +{ + oid => '4557', descr => 'I/O', + proname => 'interval_print', prorettype => 'void', proargtypes => 'interval internal', + prosrc => 'interval_print'}, + +{ + oid => '4558', descr => 'I/O', + proname => 'numeric_print', prorettype => 'void', proargtypes => 'numeric internal', + prosrc => 'numeric_print'}, + +{ + oid => '4559', descr => 'I/O', + proname => 'charprint', prorettype => 'void', proargtypes => 'char internal', + prosrc => 'charprint'}, + +{ + oid => '4560', descr => 'I/O', + proname => 'bpcharprint', prorettype => 'void', proargtypes => 'bpchar internal', + prosrc => 'bpcharprint'}, + +{ + oid => '4561', descr => 'I/O', + proname => 'varcharprint', prorettype => 'void', proargtypes => 'varchar internal', + prosrc => 'varcharprint'}, + ] diff --git a/src/include/lib/stringinfo.h b/src/include/lib/stringinfo.h index 079652c8ce4..6318fe8be5b 100644 --- a/src/include/lib/stringinfo.h +++ b/src/include/lib/stringinfo.h @@ -267,4 +267,23 @@ extern void enlargeStringInfo(StringInfo str, int needed); */ extern void destroyStringInfo(StringInfo str); +/* + * outString - The StringInfo used in type specific out function. + */ +static inline char * +outStringReserveLen(StringInfo buf, uint32 data_len) +{ + /* sizeof(uint32) is for storing the data_len itself. */ + enlargeStringInfo(buf, sizeof(uint32) + data_len); + return buf->data + buf->len + sizeof(uint32); +} + +/* define outStringCompletePhase as macro to avoid including pg_bswap.h */ +#define outStringCompletePhase(buf, data_len) \ +{ \ + *(uint32 *)(buf->data + buf->len) = pg_hton32(data_len); \ + buf->len += sizeof(uint32) + data_len; \ +} + + #endif /* STRINGINFO_H */ diff --git a/src/include/utils/date.h b/src/include/utils/date.h index 6063810891e..3a2fe34061e 100644 --- a/src/include/utils/date.h +++ b/src/include/utils/date.h @@ -111,7 +111,7 @@ extern DateADT timestamptz2date_safe(TimestampTz timestamp, Node *escontext); extern int32 date_cmp_timestamp_internal(DateADT dateVal, Timestamp dt2); extern int32 date_cmp_timestamptz_internal(DateADT dateVal, TimestampTz dt2); -extern void EncodeSpecialDate(DateADT dt, char *str); +extern int EncodeSpecialDate(DateADT dt, char *str); extern DateADT GetSQLCurrentDate(void); extern TimeTzADT *GetSQLCurrentTime(int32 typmod); extern TimeADT GetSQLLocalTime(int32 typmod); diff --git a/src/include/utils/datetime.h b/src/include/utils/datetime.h index f77c6acd8b6..a6754195e0b 100644 --- a/src/include/utils/datetime.h +++ b/src/include/utils/datetime.h @@ -330,11 +330,11 @@ extern int DetermineTimeZoneAbbrevOffset(struct pg_tm *tm, const char *abbr, pg_ extern int DetermineTimeZoneAbbrevOffsetTS(TimestampTz ts, const char *abbr, pg_tz *tzp, int *isdst); -extern void EncodeDateOnly(struct pg_tm *tm, int style, char *str); -extern void EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, char *str); -extern void EncodeDateTime(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, const char *tzn, int style, char *str); +extern int EncodeDateOnly(struct pg_tm *tm, int style, char *str); +extern int EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, char *str); +extern int EncodeDateTime(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, const char *tzn, int style, char *str); extern void EncodeInterval(struct pg_itm *itm, int style, char *str); -extern void EncodeSpecialTimestamp(Timestamp dt, char *str); +extern int EncodeSpecialTimestamp(Timestamp dt, char *str); extern int ValidateDate(int fmask, bool isjulian, bool is2digits, bool bc, struct pg_tm *tm); -- 2.43.0 --=-=-=-- ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-09-12 10:03 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-12-18 16:19 [PATCH v43 1/2] Subscripting for jsonb Dmitrii Dolgov <[email protected]> 2024-09-12 10:03 [PATCH v20260912 4/4] Make printtup a bit faster (intermediate state). Andy Fan <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox