public inbox for [email protected]
help / color / mirror / Atom feedRe: Extract numeric filed in JSONB more effectively
35+ messages / 5 participants
[nested] [flat]
* Re: Extract numeric filed in JSONB more effectively
@ 2023-08-16 16:32 jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: jian he @ 2023-08-16 16:32 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; Chapman Flack <[email protected]>; pgsql-hackers
On Wed, Aug 16, 2023 at 2:28 PM Andy Fan <[email protected]> wrote:
>
> update with the correct patch..
regression=# select proname, pg_catalog.pg_get_function_arguments(oid)
from pg_proc
where proname = 'jsonb_extract_path_type';
proname | pg_get_function_arguments
-------------------------+--------------------------------------------------------------------
jsonb_extract_path_type | from_json jsonb, VARIADIC path_elems
text[], target_oid anyelement
(1 row)
VARIADIC should be the last argument?
select jsonb_array_element_type(jsonb'[1231]',0, null::int);
now return null.
Should it return 1231?
regression=# select jsonb_array_element_type(jsonb'1231',0, 1::int);
jsonb_array_element_type
--------------------------
1231
(1 row)
not sure if it's ok. if you think it's not ok then:
+ if (!JB_ROOT_IS_ARRAY(jb))
+PG_RETURN_NULL();
change to
+if (JB_ROOT_IS_SCALAR(jb) || !JB_ROOT_IS_ARRAY(jb))
+PG_RETURN_NULL();
select jsonb_array_element_type(jsonb'[1231]',0, '1'::jsonb);
will crash, because jsonb_array_element_type call
cast_jsonbvalue_to_type then in switch case, it will go to
default part. in default part you have Assert(false);
also in cast_jsonbvalue_to_type, PG_RETURN_POINTER(NULL) code won't be reached.
in jsonb_cast_support function. you already have
!jsonb_cast_is_optimized(fexpr->funcresulttype)). then in the default
branch of cast_jsonbvalue_to_type, you can just elog(error, "can only
cast to xxx type"). jsonb_array_element_type, jsonb_object_field_type,
third argument is anyelement. so targetOid can be any datatype's oid.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
@ 2023-08-17 09:07 ` Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-17 09:07 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; Chapman Flack <[email protected]>; pgsql-hackers
Hi jian:
On Thu, Aug 17, 2023 at 12:32 AM jian he <[email protected]>
wrote:
> On Wed, Aug 16, 2023 at 2:28 PM Andy Fan <[email protected]> wrote:
> >
> > update with the correct patch..
>
> regression=# select proname, pg_catalog.pg_get_function_arguments(oid)
> from pg_proc
> where proname = 'jsonb_extract_path_type';
> proname | pg_get_function_arguments
>
> -------------------------+--------------------------------------------------------------------
> jsonb_extract_path_type | from_json jsonb, VARIADIC path_elems
> text[], target_oid anyelement
> (1 row)
>
> VARIADIC should be the last argument?
>
Currently if users call this function directly(usually I don't think
so), they will get something wrong. This issue is fixed in the
v9 version. To keep the consistency among all the functions,
I moved the 'target_type anyelement' to the 1st argument.
Thanks for the report!
> select jsonb_array_element_type(jsonb'[1231]',0, null::int);
> now return null.
> Should it return 1231?
>
No, this is by design. the function is declared as strict, so
any NULL inputs yield a NULL output. That's just what we
talked above (the markDummyConst section). I don't
think this should be addressed.
> select jsonb_array_element_type(jsonb'[1231]',0, '1'::jsonb);
> will crash
>
OK, looks I didn't pay enough attention to the 'user directly call
jsonb_xx_type' function, so I changed the code in v9 based on
your suggestion.
Thanks for the review, v9 attached!
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] test.sql (4.4K, 3-test.sql)
download
[application/octet-stream] v9-0001-optimize-casting-jsonb-to-a-given-type.patch (31.7K, 4-v9-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From c4b1ae13a0f4ba28972835ffa4c9850e2e0dbda6 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Wed, 16 Aug 2023 14:04:27 +0800
Subject: [PATCH v9] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/nodes/makefuncs.c | 30 +++++
src/backend/utils/adt/jsonb.c | 177 +++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 115 +++++++++++-------
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 32 +++--
src/include/nodes/makefuncs.h | 2 +
src/include/utils/jsonb.h | 1 +
src/test/regress/expected/jsonb.out | 178 +++++++++++++++++-----------
src/test/regress/sql/jsonb.sql | 56 ++++++---
9 files changed, 462 insertions(+), 131 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 0e7e6e46d94..9cb9178f01a 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -20,6 +20,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "utils/errcodes.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
@@ -352,6 +353,35 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
+/*
+ * makeDummyConst
+ * create a Const node with the specified type/typmod.
+ *
+ * This is a convenience routine to create a Const which only the
+ * type is interesting but make sure the value is accessible.
+ */
+Const *
+makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
+{
+ int16 typLen;
+ bool typByVal;
+ Const *c;
+ Datum val = 0;
+
+
+ get_typlenbyval(consttype, &typLen, &typByVal);
+
+ if (consttype == NUMERICOID)
+ val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
+ else if (!typByVal)
+ elog(ERROR, "create dummy const for type %u is not supported.", consttype);
+
+ /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
+ c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
+
+ return c;
+}
+
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..148c1e2e195 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,14 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2041,180 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc(sizeof(FuncExpr));
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_TYPE;
+
+ if (OidIsValid(new_func_id))
+ {
+ Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ fexpr->funcid = new_func_id;
+ fexpr->args = opexpr->args;
+ fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ }
+
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
+{
+ switch(targetOid)
+ {
+ Datum retValue;
+
+ case BOOLOID:
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "bool");
+ PG_RETURN_BOOL(v->val.boolean);
+
+ case NUMERICOID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "numeric");
+ PG_RETURN_NUMERIC(v->val.numeric);
+ case INT2OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "smallint");
+ retValue = DirectFunctionCall1(numeric_int2,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+ case INT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "integer");
+ retValue = DirectFunctionCall1(numeric_int4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case INT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "bigint");
+ retValue = DirectFunctionCall1(numeric_int8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "real");
+ retValue = DirectFunctionCall1(numeric_float4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "double precision");
+ retValue = DirectFunctionCall1(numeric_float8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ default:
+ elog(ERROR, "cast jsonb to type %u is not allowed", targetOid);
+ break;
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+jsonb_object_field_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ text *key = PG_GETARG_TEXT_PP(2);
+
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
+Datum
+jsonb_array_element_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ int element = PG_GETARG_INT32(2);
+
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..bb4ca807d74 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,40 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1551,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1605,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1626,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1638,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1663,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index f507b49bb28..8a896f9aad2 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202307261
+#define CATALOG_VERSION_NO 202308171
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989a..b6844537529 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4575,25 +4575,26 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prosupport => 'jsonb_cast_support',
+ prorettype => 'numeric', proargtypes => 'jsonb',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9928,6 +9929,13 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
+ proname => 'jsonb_object_field_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ prosrc => 'jsonb_object_field_type'},
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -9936,6 +9944,10 @@
proname => 'jsonb_array_element_text', prorettype => 'text',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
+{ oid => '4549', descr => 'cast an array element to given type',
+ proname => 'jsonb_array_element_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
proargtypes => 'jsonb _text', proallargtypes => '{jsonb,_text}',
@@ -9947,6 +9959,12 @@
proallargtypes => '{jsonb,_text}', proargmodes => '{i,v}',
proargnames => '{from_json,path_elems}',
prosrc => 'jsonb_extract_path_text' },
+{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
+ proname => 'jsonb_extract_path_type', provariadic => 'text',
+ prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
+ proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ proargnames => '{target_type,from_json,path_elems}',
+ prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
proname => 'jsonb_array_elements', prorows => '100', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 31807030055..cfbe5b26196 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,6 +58,8 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
+extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
+
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index 649a1644f24..532225314a9 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -435,5 +435,6 @@ extern Datum jsonb_build_object_worker(int nargs, Datum *args, bool *nulls,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null);
+extern Datum cast_jsonbvalue_to_type(JsonbValue *v, Oid target_oid);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..12daacb3b80 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,6 +457,7 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
@@ -501,10 +502,25 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
val2
(1 row)
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+ ?column? | ?column?
+----------+------------
+ | "a scalar"
+(1 row)
+
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+-----------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
?column?
----------
-
+ 2
(1 row)
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
@@ -1786,6 +1802,12 @@ select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
{"a": {"b": {"c": "foo"}}}
(1 row)
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
+ int2 | ?column? | int2
+------+----------+------
+ 2 | |
+(1 row)
+
select '[1,2,3]'::jsonb #> '{}';
?column?
-----------
@@ -3537,6 +3559,24 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.testjsonb
+ Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+(2 rows)
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
jsonb
@@ -5471,107 +5511,113 @@ select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
(1 row)
-- casts
-select 'true'::jsonb::bool;
- bool
-------
- t
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
+ bool | bool
+------+------
+ t | t
(1 row)
select '[]'::jsonb::bool;
ERROR: cannot cast jsonb array to type boolean
-select '1.0'::jsonb::float;
- float8
---------
- 1
+select ('{"a": []}'::jsonb->'a')::bool;
+ERROR: cannot cast jsonb array to type boolean
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
+ float8 | float8
+--------+--------
+ 1 | 1
(1 row)
select '[1.0]'::jsonb::float;
ERROR: cannot cast jsonb array to type double precision
-select '12345'::jsonb::int4;
- int4
--------
- 12345
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+ERROR: cannot cast jsonb array to type double precision
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
select '"hello"'::jsonb::int4;
ERROR: cannot cast jsonb string to type integer
-select '12345'::jsonb::numeric;
- numeric
----------
- 12345
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+ERROR: cannot cast jsonb string to type integer
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
+ numeric | numeric
+---------+---------
+ 12345 | 12345
(1 row)
select '{}'::jsonb::numeric;
ERROR: cannot cast jsonb object to type numeric
-select '12345.05'::jsonb::numeric;
- numeric
-----------
- 12345.05
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+ numeric | numeric
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float4;
- float4
-----------
- 12345.05
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+ float4 | float4
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float8;
- float8
-----------
- 12345.05
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+ float8 | float8
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::int2;
- int2
--------
- 12345
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int4;
- int4
--------
- 12345
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int8;
- int8
--------
- 12345
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
- numeric
-------------------------------------------------------
- 12345.0000000000000000000000000000000000000000000005
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+ numeric | numeric
+------------------------------------------------------+------------------------------------------------------
+ 12345.0000000000000000000000000000000000000000000005 | 12345.0000000000000000000000000000000000000000000005
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
- float4
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+ float4 | float4
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
- float8
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+ float8 | float8
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
- int2
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
- int4
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
- int8
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..8634d154efe 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,6 +154,7 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
@@ -166,7 +167,10 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'array';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 9 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'object';
@@ -491,6 +495,7 @@ SELECT '{"f2":["f3",1],"f4":{"f5":99,"f6":"stringy"}}'::jsonb#>>array['f2','1'];
-- corner cases for same
select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
select '[1,2,3]'::jsonb #> '{}';
select '"foo"'::jsonb #> '{}';
select '42'::jsonb #> '{}';
@@ -939,6 +944,19 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
SELECT '{"ff":{"a":12,"b":16},"qq":123}'::jsonb;
@@ -1496,23 +1514,27 @@ select ts_headline('{}'::jsonb, tsquery('aaa & bbb'));
select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
-- casts
-select 'true'::jsonb::bool;
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
select '[]'::jsonb::bool;
-select '1.0'::jsonb::float;
+select ('{"a": []}'::jsonb->'a')::bool;
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
select '[1.0]'::jsonb::float;
-select '12345'::jsonb::int4;
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
select '"hello"'::jsonb::int4;
-select '12345'::jsonb::numeric;
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
select '{}'::jsonb::numeric;
-select '12345.05'::jsonb::numeric;
-select '12345.05'::jsonb::float4;
-select '12345.05'::jsonb::float8;
-select '12345.05'::jsonb::int2;
-select '12345.05'::jsonb::int4;
-select '12345.05'::jsonb::int8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-17 20:30 ` Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-17 20:30 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-17 05:07, Andy Fan wrote:
> Thanks for the review, v9 attached!
From the earliest iterations of this patch, I seem to recall
a couple of designs being considered:
In one, the type-specific cast function would only be internally
usable, would take a type oid as an extra parameter (supplied in
the SupportRequestSimplify rewriting), and would have to be
declared with some nonspecific return type; 'internal' was
mentioned.
The idea of an 'internal' return type with no 'internal' parameter
was quickly and rightly shot down. But it would have seemed to me
enough to address that objection by using 'internal' also in its
parameter list. I could imagine a function declared with two
'internal' parameters, one understood to be a JsonbValue and one
understood to be a type oid, and an 'internal' result, treated in
the rewritten expression tree as binary-coercible to the desired
result.
Admittedly, I have not tried to implement that myself to see
what unexpected roadblocks might exist on that path. Perhaps
there are parts of that rewriting that no existing node type
can represent? Someone more familiar with those corners of
PostgreSQL may immediately see other difficulties I do not.
But I have the sense that that approach was abandoned early, in
favor of the current approach using user-visible polymorphic
types, and supplying typed dummy constants for use in the
resolution of those types, with a new function introduced to create
said dummy constants, including allocation and input conversion
in the case of numeric, just so said dummy constants can be
passed into functions that have no use for them other than to
call get_fn_expr_argtype to recover the type oid, which was the
only thing needed in the first place.
Compared to the initial direction I thought this was going,
none of that strikes me as better.
Nothing makes my opinion authoritative here, and there may
indeed be reasons it is better, known to others more familiar
with that code than I am. But it bugs me.
If the obstacles to the earlier approach came down to needing
a new type of expression node, something like "assertion of
'internal'-to-foo binary coercibility, vouched by a prosupport
function", would that be a bad thing?
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-18 01:14 ` Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-18 01:14 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hi Chapman,
Thanks for the review!
The idea of an 'internal' return type with no 'internal' parameter
> was quickly and rightly shot down.
Yes, it mainly breaks the type-safety system. Parser need to know
the result type, so PG defines the rule like this:
anyelement fn(anyment in);
if the exprType(in) in the query tree is X, then PG would think fn
return type X. that's why we have to have an anyelement in the
input.
> But it would have seemed to me
> enough to address that objection by using 'internal' also in its
> parameter list. I could imagine a function declared with two
> 'internal' parameters, one understood to be a JsonbValue and one
> understood to be a type oid, and an 'internal' result, treated in
> the rewritten expression tree as binary-coercible to the desired
> result.
>
I have some trouble understanding this. are you saying something
like:
internal fn(internal jsonValue, internal typeOid)?
If so, would it break the type-safety system? And I'm not pretty
sure the 'binary-coercible' here. is it same as the 'binary-coercible'
in "timestamp is not binary coercible with timestamptz since..."?
I have a strong feeling that I think I misunderstood you here.
> Perhaps there are parts of that rewriting that no existing node type
> can represent?
>
I didn't understand this as well:(:(
But I have the sense that that approach was abandoned early, in
> favor of the current approach using user-visible polymorphic
> types, and supplying typed dummy constants for use in the
> resolution of those types, with a new function introduced to create
> said dummy constants, including allocation and input conversion
> in the case of numeric, just so said dummy constants can be
> passed into functions that have no use for them other than to
> call get_fn_expr_argtype to recover the type oid, which was the
> only thing needed in the first place.
Yes, but if we follow the type-safety system, we can't simply input
a Oid targetOid, then there are some more considerations here:
a). we can't use the makeNullConst because jsonb_xxx_type is
strict, so if we have NULL constant input here, the PG system
will return NULL directly. b). Not only the type oid is the thing
We are interested in the const.constvalue is as well since
'explain select xxxx' to access it to show it as a string.
Datum(0) as the constvalue will crash in this sense. That's why
makeDummyConst was introduced.
> something like "assertion of
> 'internal'-to-foo binary coercibility, vouched by a prosupport
> function", would that be a bad thing?
>
I can't follow this as well. Could you provide the function prototype
here?
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-18 02:55 ` Chapman Flack <[email protected]>
2023-08-18 05:02 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Chapman Flack @ 2023-08-18 02:55 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-17 21:14, Andy Fan wrote:
>> The idea of an 'internal' return type with no 'internal' parameter
>> was quickly and rightly shot down.
>
> Yes, it mainly breaks the type-safety system. Parser need to know
> the result type, so PG defines the rule like this:
Well, the reason "internal return type with no internal parameter type"
was shot down was more specific: if there is such a function in the
catalog, an SQL user can call it, and then its return type is a value
typed 'internal', and with that the SQL user could call other
functions with 'internal' parameters, and that's what breaks type
safety. The specific problem is not having at least one 'internal'
input parameter.
There are lots of functions in the catalog with internal return type
(I count 111). They are not inherently bad; the rule is simply that
each one also needs at least one IN parameter typed internal, to
make sure it can't be directly called from SQL.
> anyelement fn(anyment in);
>
> if the exprType(in) in the query tree is X, then PG would think fn
> return type X. that's why we have to have an anyelement in the
> input.
That's a consequence of the choice to have anyelement as the return
type, though. A different choice wouldn't have that consequence.
> I have some trouble understanding this. are you saying something
> like:
>
> internal fn(internal jsonValue, internal typeOid)?
>
> If so, would it break the type-safety system?
That is what I'm saying, and it doesn't break type safety at the
SQL level, because as long as it has parameters declared internal,
no SQL can ever call it. So it can only appear in an expression
tree because your SupportRequestSimplify put it there properly
typed, after the SQL query was parsed but before evaluation.
The thing about 'internal' is it doesn't represent any specific
type, it doesn't necessarily represent the same type every time it
is mentioned, and it often means something that isn't a cataloged
type at all, such as a pointer to some kind of struct. There must be
documentation explaining what it has to be. For example, your
jsonb_cast_support function has an 'internal' parameter and
'internal' return type. From the specification for support
functions, you know the 'internal' for the parameter type means
"one of the Node structs in supportnodes.h", and the 'internal'
for the return type means "an expression tree semantically
equivalent to the FuncExpr".
So, in addition to declaring
internal fn(internal jsonValue, internal typeOid), you would
have to write a clear spec that jsonValue has to be a JsonbValue,
typeOid has to be something you can call DatumGetObjectId on,
and the return value should be a Datum in proper form
corresponding to typeOid. And, of course, generate the expression
tree so all of that is true when it's evaluated.
>> Perhaps there are parts of that rewriting that no existing node type
>> can represent?
The description above was in broad strokes. Because I haven't
tried to implement this, I don't know whether some roadblock would
appear, such as, is it hard to make a Const node of type internal
and containing an oid? Or, what sort of node must be inserted to
clarify that the 'internal' return is actually a Datum of the
expected type? By construction, we know that it is, but how to
make that explicit in the expression tree?
> a). we can't use the makeNullConst because jsonb_xxx_type is
> strict, so if we have NULL constant input here, the PG system
> will return NULL directly. b). Not only the type oid is the thing
> We are interested in the const.constvalue is as well since
> 'explain select xxxx' to access it to show it as a string.
> Datum(0) as the constvalue will crash in this sense. That's why
> makeDummyConst was introduced.
Again, all of that complication stems from the choice to use the
anyelement return type and rely on polymorphic type resolution
to figure the oid out, when we already have the oid to begin with
and the oid is all we want.
>> something like "assertion of
>> 'internal'-to-foo binary coercibility, vouched by a prosupport
>> function", would that be a bad thing?
>
> I can't follow this as well.
That was just another way of saying what I was getting at above
about what's needed in the expression tree to indicate that the
'internal' produced by this function is, in fact, really a bool
(or whatever). We know that it is, but perhaps the expression
tree will be considered ill-formed without a node that says so.
A node representing a no-op, binary conversion would suffice,
but is there already a node that's allowed to represent an
internal-to-bool no-op cast?
If there isn't, one might have to be invented. So it might be that
if we go down the "use polymorphic resolution" road, we have to
invent dummy Consts, and down the "internal" road we also have to
invent something, like the "no-op cast considered correct because
a SupportRequestSimplify function put it here" node.
If it came down to having to invent one of those things or the
other, I'd think the latter more directly captures what we really
want to do.
(And I'm not even sure anything has to be invented. If there's an
existing node for no-op binary casts, I think I'd first try
putting that there and see if anything complains.)
If this thread is being followed by others more familiar with
the relevant code or who see obvious problems I'm missing,
please chime in!
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-18 05:02 ` jian he <[email protected]>
1 sibling, 0 replies; 35+ messages in thread
From: jian he @ 2023-08-18 05:02 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: Andy Fan <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Fri, Aug 18, 2023 at 10:55 AM Chapman Flack <[email protected]> wrote:
>
>
> Again, all of that complication stems from the choice to use the
> anyelement return type and rely on polymorphic type resolution
> to figure the oid out, when we already have the oid to begin with
> and the oid is all we want.
>
you want jsonb_object_field_type(internal, jsonb, text)? because on
sql level, it's safe.
The return data type is determined when we are in jsonb_cast_support.
we just need to pass the {return data type} information to the next
function: jsonb_object_field_type.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-18 07:41 ` Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-18 07:41 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> because as long as it has parameters declared internal,
> no SQL can ever call it.
I was confused about the difference between anyelement and
internal, and I want to know a way to create a function which
is disallowed to be called by the user. Your above words
resolved two questions of mine!
So it can only appear in an expression
> tree because your SupportRequestSimplify put it there properly
> typed, after the SQL query was parsed but before evaluation.
>
> The thing about 'internal' is it doesn't represent any specific
> type, it doesn't necessarily represent the same type every time it
> is mentioned, and it often means something that isn't a cataloged
> type at all, such as a pointer to some kind of struct.
I should have noticed this during the study planner support function,
but highlighting this is pretty amazing.
> If there isn't, one might have to be invented. So it might be that
> if we go down the "use polymorphic resolution" road, we have to
> invent dummy Consts, and down the "internal" road we also have to
> invent something.
I think you might already feel that putting an internal function
into an expression would cause something wrong. I just have
a quick hack on this, and crash happens at the simplest case.
If something already exists to fix this, I am inclined
to use 'internal', but I didn't find the way. I'm thinking if we
should clarify "internal" should only be used internally and
should never be used in expression by design?
> (And I'm not even sure anything has to be invented. If there's an
> existing node for no-op binary casts, I think I'd first try
> putting that there and see if anything complains.)
>
> If this thread is being followed by others more familiar with
> the relevant code or who see obvious problems I'm missing,
> please chime in!
>
Thank you wise & modest gentleman, I would really hope Tom can
chime in at this time.
In general, the current decision we need to make is shall we use
'internal' or 'anyelement' to present the target OID. the internal way
would be more straight but have troubles to be in the expression tree.
the 'anyelement' way is compatible with expression, but it introduces
the makeDummyConst overhead and I'm not pretty sure it is a correct
implementation in makeDummyConst. see the XXX part.
+/*
+ * makeDummyConst
+ * create a Const node with the specified type/typmod.
+ *
+ * This is a convenience routine to create a Const which only the
+ * type is interested but make sure the value is accessible.
+ */
+Const *
+makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
+{
+ int16 typLen;
+ bool typByVal;
+ Const *c;
+ Datum val = 0;
+
+
+ get_typlenbyval(consttype, &typLen, &typByVal);
+
+ if (consttype == NUMERICOID)
+ val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
+ else if (!typByVal)
+ elog(ERROR, "create dummy const for type %u is not
supported.", consttype);
+
+ /* XXX: here I assume constvalue=0 is accessible for const by value
type.*/
+ c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false,
typByVal);
+
+ return c;
+}
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] 0001-convert-anyelement-to-internal.patch (4.5K, 3-0001-convert-anyelement-to-internal.patch)
download | inline diff:
From edca83956c65b060437cab55b50dcfe76414065b Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Fri, 18 Aug 2023 15:38:50 +0800
Subject: [PATCH] convert anyelement to internal.
---
src/backend/utils/adt/jsonb.c | 9 +++++----
src/backend/utils/adt/jsonfuncs.c | 2 +-
src/include/catalog/pg_proc.dat | 12 ++++++------
3 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 148c1e2e195..7c1faa3d7f7 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -2091,10 +2091,11 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
if (OidIsValid(new_func_id))
{
- Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ // Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ int64 target_typ = fexpr->funcresulttype;
fexpr->funcid = new_func_id;
fexpr->args = opexpr->args;
- fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ fexpr->args = list_insert_nth(fexpr->args, 0, (void *) target_typ);
}
PG_RETURN_POINTER(fexpr);
@@ -2164,7 +2165,7 @@ cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
Datum
jsonb_object_field_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = DatumGetObjectId(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
text *key = PG_GETARG_TEXT_PP(2);
@@ -2188,7 +2189,7 @@ jsonb_object_field_type(PG_FUNCTION_ARGS)
Datum
jsonb_array_element_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = DatumGetObjectId(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
int element = PG_GETARG_INT32(2);
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index bb4ca807d74..54138258bfc 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -1477,7 +1477,7 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = DatumGetObjectId(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b6844537529..66d1af71586 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9930,8 +9930,8 @@
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
- proname => 'jsonb_object_field_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ proname => 'jsonb_object_field_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb text', proargnames => '{target_type, from_json, field_name}',
prosrc => 'jsonb_object_field_type'},
{ oid => '3814', descr => 'planner support for numeric(jsonb)',
proname => 'jsonb_cast_support', prorettype => 'internal',
@@ -9945,8 +9945,8 @@
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
{ oid => '4549', descr => 'cast an array element to given type',
- proname => 'jsonb_array_element_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ proname => 'jsonb_array_element_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb int4', proargnames => '{target_type, from_json, element_index}',
prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
@@ -9961,8 +9961,8 @@
prosrc => 'jsonb_extract_path_text' },
{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
proname => 'jsonb_extract_path_type', provariadic => 'text',
- prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
- proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ prorettype => 'internal', proargtypes => 'internal jsonb _text',
+ proallargtypes => '{internal,jsonb,_text}', proargmodes => '{i,i,v}',
proargnames => '{target_type,from_json,path_elems}',
prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
--
2.21.0
[application/octet-stream] v9-0001-optimize-casting-jsonb-to-a-given-type.patch (31.7K, 4-v9-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From c4b1ae13a0f4ba28972835ffa4c9850e2e0dbda6 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Wed, 16 Aug 2023 14:04:27 +0800
Subject: [PATCH v9] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/nodes/makefuncs.c | 30 +++++
src/backend/utils/adt/jsonb.c | 177 +++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 115 +++++++++++-------
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 32 +++--
src/include/nodes/makefuncs.h | 2 +
src/include/utils/jsonb.h | 1 +
src/test/regress/expected/jsonb.out | 178 +++++++++++++++++-----------
src/test/regress/sql/jsonb.sql | 56 ++++++---
9 files changed, 462 insertions(+), 131 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 0e7e6e46d94..9cb9178f01a 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -20,6 +20,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "utils/errcodes.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
@@ -352,6 +353,35 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
+/*
+ * makeDummyConst
+ * create a Const node with the specified type/typmod.
+ *
+ * This is a convenience routine to create a Const which only the
+ * type is interesting but make sure the value is accessible.
+ */
+Const *
+makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
+{
+ int16 typLen;
+ bool typByVal;
+ Const *c;
+ Datum val = 0;
+
+
+ get_typlenbyval(consttype, &typLen, &typByVal);
+
+ if (consttype == NUMERICOID)
+ val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
+ else if (!typByVal)
+ elog(ERROR, "create dummy const for type %u is not supported.", consttype);
+
+ /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
+ c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
+
+ return c;
+}
+
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..148c1e2e195 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,14 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2041,180 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc(sizeof(FuncExpr));
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_TYPE;
+
+ if (OidIsValid(new_func_id))
+ {
+ Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ fexpr->funcid = new_func_id;
+ fexpr->args = opexpr->args;
+ fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ }
+
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
+{
+ switch(targetOid)
+ {
+ Datum retValue;
+
+ case BOOLOID:
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "bool");
+ PG_RETURN_BOOL(v->val.boolean);
+
+ case NUMERICOID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "numeric");
+ PG_RETURN_NUMERIC(v->val.numeric);
+ case INT2OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "smallint");
+ retValue = DirectFunctionCall1(numeric_int2,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+ case INT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "integer");
+ retValue = DirectFunctionCall1(numeric_int4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case INT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "bigint");
+ retValue = DirectFunctionCall1(numeric_int8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "real");
+ retValue = DirectFunctionCall1(numeric_float4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "double precision");
+ retValue = DirectFunctionCall1(numeric_float8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ default:
+ elog(ERROR, "cast jsonb to type %u is not allowed", targetOid);
+ break;
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+jsonb_object_field_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ text *key = PG_GETARG_TEXT_PP(2);
+
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
+Datum
+jsonb_array_element_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ int element = PG_GETARG_INT32(2);
+
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..bb4ca807d74 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,40 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1551,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1605,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1626,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1638,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1663,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index f507b49bb28..8a896f9aad2 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202307261
+#define CATALOG_VERSION_NO 202308171
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989a..b6844537529 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4575,25 +4575,26 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prosupport => 'jsonb_cast_support',
+ prorettype => 'numeric', proargtypes => 'jsonb',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9928,6 +9929,13 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
+ proname => 'jsonb_object_field_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ prosrc => 'jsonb_object_field_type'},
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -9936,6 +9944,10 @@
proname => 'jsonb_array_element_text', prorettype => 'text',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
+{ oid => '4549', descr => 'cast an array element to given type',
+ proname => 'jsonb_array_element_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
proargtypes => 'jsonb _text', proallargtypes => '{jsonb,_text}',
@@ -9947,6 +9959,12 @@
proallargtypes => '{jsonb,_text}', proargmodes => '{i,v}',
proargnames => '{from_json,path_elems}',
prosrc => 'jsonb_extract_path_text' },
+{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
+ proname => 'jsonb_extract_path_type', provariadic => 'text',
+ prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
+ proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ proargnames => '{target_type,from_json,path_elems}',
+ prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
proname => 'jsonb_array_elements', prorows => '100', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 31807030055..cfbe5b26196 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,6 +58,8 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
+extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
+
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index 649a1644f24..532225314a9 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -435,5 +435,6 @@ extern Datum jsonb_build_object_worker(int nargs, Datum *args, bool *nulls,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null);
+extern Datum cast_jsonbvalue_to_type(JsonbValue *v, Oid target_oid);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..12daacb3b80 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,6 +457,7 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
@@ -501,10 +502,25 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
val2
(1 row)
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+ ?column? | ?column?
+----------+------------
+ | "a scalar"
+(1 row)
+
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+-----------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
?column?
----------
-
+ 2
(1 row)
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
@@ -1786,6 +1802,12 @@ select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
{"a": {"b": {"c": "foo"}}}
(1 row)
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
+ int2 | ?column? | int2
+------+----------+------
+ 2 | |
+(1 row)
+
select '[1,2,3]'::jsonb #> '{}';
?column?
-----------
@@ -3537,6 +3559,24 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.testjsonb
+ Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+(2 rows)
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
jsonb
@@ -5471,107 +5511,113 @@ select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
(1 row)
-- casts
-select 'true'::jsonb::bool;
- bool
-------
- t
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
+ bool | bool
+------+------
+ t | t
(1 row)
select '[]'::jsonb::bool;
ERROR: cannot cast jsonb array to type boolean
-select '1.0'::jsonb::float;
- float8
---------
- 1
+select ('{"a": []}'::jsonb->'a')::bool;
+ERROR: cannot cast jsonb array to type boolean
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
+ float8 | float8
+--------+--------
+ 1 | 1
(1 row)
select '[1.0]'::jsonb::float;
ERROR: cannot cast jsonb array to type double precision
-select '12345'::jsonb::int4;
- int4
--------
- 12345
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+ERROR: cannot cast jsonb array to type double precision
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
select '"hello"'::jsonb::int4;
ERROR: cannot cast jsonb string to type integer
-select '12345'::jsonb::numeric;
- numeric
----------
- 12345
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+ERROR: cannot cast jsonb string to type integer
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
+ numeric | numeric
+---------+---------
+ 12345 | 12345
(1 row)
select '{}'::jsonb::numeric;
ERROR: cannot cast jsonb object to type numeric
-select '12345.05'::jsonb::numeric;
- numeric
-----------
- 12345.05
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+ numeric | numeric
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float4;
- float4
-----------
- 12345.05
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+ float4 | float4
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float8;
- float8
-----------
- 12345.05
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+ float8 | float8
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::int2;
- int2
--------
- 12345
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int4;
- int4
--------
- 12345
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int8;
- int8
--------
- 12345
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
- numeric
-------------------------------------------------------
- 12345.0000000000000000000000000000000000000000000005
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+ numeric | numeric
+------------------------------------------------------+------------------------------------------------------
+ 12345.0000000000000000000000000000000000000000000005 | 12345.0000000000000000000000000000000000000000000005
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
- float4
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+ float4 | float4
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
- float8
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+ float8 | float8
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
- int2
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
- int4
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
- int8
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..8634d154efe 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,6 +154,7 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
@@ -166,7 +167,10 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'array';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 9 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'object';
@@ -491,6 +495,7 @@ SELECT '{"f2":["f3",1],"f4":{"f5":99,"f6":"stringy"}}'::jsonb#>>array['f2','1'];
-- corner cases for same
select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
select '[1,2,3]'::jsonb #> '{}';
select '"foo"'::jsonb #> '{}';
select '42'::jsonb #> '{}';
@@ -939,6 +944,19 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
SELECT '{"ff":{"a":12,"b":16},"qq":123}'::jsonb;
@@ -1496,23 +1514,27 @@ select ts_headline('{}'::jsonb, tsquery('aaa & bbb'));
select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
-- casts
-select 'true'::jsonb::bool;
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
select '[]'::jsonb::bool;
-select '1.0'::jsonb::float;
+select ('{"a": []}'::jsonb->'a')::bool;
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
select '[1.0]'::jsonb::float;
-select '12345'::jsonb::int4;
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
select '"hello"'::jsonb::int4;
-select '12345'::jsonb::numeric;
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
select '{}'::jsonb::numeric;
-select '12345.05'::jsonb::numeric;
-select '12345.05'::jsonb::float4;
-select '12345.05'::jsonb::float8;
-select '12345.05'::jsonb::int2;
-select '12345.05'::jsonb::int4;
-select '12345.05'::jsonb::int8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-18 18:50 ` Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-18 18:50 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-18 03:41, Andy Fan wrote:
> I just have
> a quick hack on this, and crash happens at the simplest case.
If I build from this patch, this test:
SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE
json_type = 'scalarint';
fails like this:
Program received signal SIGSEGV, Segmentation fault.
convert_saop_to_hashed_saop_walker (node=0x17, context=0x0)
at
/var/tmp/nohome/pgbuildh/../postgresql/src/backend/optimizer/util/clauses.c:2215
2215 if (IsA(node, ScalarArrayOpExpr))
(gdb) p node
$1 = (Node *) 0x17
So the optimizer is looking at some node to see if it is a
ScalarArrayOpExpr, but the node has some rather weird address.
Or maybe it's not that weird. 0x17 is 23, and so is:
select 'int4'::regtype::oid;
oid
-----
23
See what happened?
+ int64 target_typ = fexpr->funcresulttype;
...
+ fexpr->args = list_insert_nth(fexpr->args, 0, (void *) target_typ);
This is inserting the desired result type oid directly as the first
thing in the list of fexpr's args.
But at the time your support function is called, nothing is being
evaluated yet. You are just manipulating a tree of expressions to
be evaluated later, and you want fexpr's first arg to be an
expression that will produce this type oid later, when it is
evaluated.
A constant would do nicely:
+ Const *target = makeConst(
INTERNALOID, -1, InvalidOid, SIZEOF_DATUM,
ObjectIdGetDatum(fexpr->funcresulttype), false, true);
+ fexpr->args = list_insert_nth(fexpr->args, 0, target);
With that change, it doesn't segfault, but it does do this:
ERROR: cast jsonb to type 0 is not allowed
and that's because of this:
+ Oid targetOid = DatumGetObjectId(0);
The DatumGetFoo(x) macros are for when you already have the Datum
(it's x) and you know it's a Foo. So this is just setting targetOid
to zero. When you want to get something from function argument 0 and
you know that's a Foo, you use a PG_GETARG_FOO(argno) macro (which
amounts to PG_GETARG_DATUM(argno) followed by DatumGetFoo.
So, with
+ Oid targetOid = PG_GETARG_OID(0);
SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE
json_type = 'scalarint';
int4 | ?column?
------+----------
2 | 2
However, EXPLAIN is sad:
ERROR: cannot display a value of type internal
and that may be where this idea runs aground.
Now, I was expecting something to complain about the result of
jsonb_array_element_type, and that didn't happen. We rewrote
a function that was supposed to be a cast to int4, and
replaced it with a function returning internal, and evaluation
happily just took that as the int4 that the next node expected.
If something had complained about that, it might have been
necessary to insert some new node above the internal-returning
function to say the result was really int4. Notice there is a
makeRelabelType() for that. (I had figured there probably was,
but didn't know its exact name.)
So it doesn't seem strictly necessary to do that, but it might
make the EXPLAIN result look better (if EXPLAIN were made to work,
of course).
Now, my guess is EXPLAIN is complaining when it sees the Const
of type internal, and doesn't know how to show that value.
Perhaps makeRelabelType is the answer there, too: what if the
Const has Oid type, so EXPLAIN can show it, and what's inserted
as the function argument is a relabel node saying it's internal?
Haven't tried that yet.
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-18 19:08 ` Chapman Flack <[email protected]>
2023-08-18 21:02 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Chapman Flack @ 2023-08-18 19:08 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-18 14:50, Chapman Flack wrote:
> Now, my guess is EXPLAIN is complaining when it sees the Const
> of type internal, and doesn't know how to show that value.
> Perhaps makeRelabelType is the answer there, too: what if the
> Const has Oid type, so EXPLAIN can show it, and what's inserted
> as the function argument is a relabel node saying it's internal?
Simply changing the Const to be of type Oid makes EXPLAIN happy,
and nothing ever says "hey, why are you passing this oid for an
arg that wants internal?". This is without adding any relabel
nodes anywhere.
Seq Scan on pg_temp.test_jsonb
Output: pg_catalog.jsonb_array_element_type('23'::oid, test_json, 0),
(test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
Nothing in that EXPLAIN output to make you think anything weird
was going on, unless you went and looked up jsonb_array_element_type
and saw that its arg0 isn't oid and its return type isn't int4.
But I don't know that adding relabel nodes wouldn't still be
the civilized thing to do.
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-18 21:02 ` Chapman Flack <[email protected]>
2023-08-21 10:58 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-18 21:02 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-18 15:08, Chapman Flack wrote:
> But I don't know that adding relabel nodes wouldn't still be
> the civilized thing to do.
Interestingly, when I relabel both places, like this:
Oid targetOid = fexpr->funcresulttype;
Const *target = makeConst(
OIDOID, -1, InvalidOid, sizeof(Oid),
ObjectIdGetDatum(targetOid), false, true);
RelabelType *rTarget = makeRelabelType((Expr *)target,
INTERNALOID, -1, InvalidOid, COERCE_IMPLICIT_CAST);
fexpr->funcid = new_func_id;
fexpr->args = opexpr->args;
fexpr->args = list_insert_nth(fexpr->args, 0, rTarget);
expr = (Expr *)makeRelabelType((Expr *)fexpr,
targetOid, -1, InvalidOid, COERCE_IMPLICIT_CAST);
}
PG_RETURN_POINTER(expr);
EXPLAIN looks like this:
Seq Scan on pg_temp.test_jsonb
Output: jsonb_array_element_type(('23'::oid)::internal, test_json,
0), (test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
With COERCE_IMPLICIT_CAST both places, the relabeling of the
function result is invisible, but the relabeling of the argument
is visible.
With the second one changed to COERCE_EXPLICIT_CAST:
Seq Scan on pg_temp.test_jsonb
Output: (jsonb_array_element_type(('23'::oid)::internal, test_json,
0))::integer, (test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
then both relabelings are visible.
I'm not sure whether one way is better than the other, or whether
it is even important to add the relabel nodes at all, as nothing
raises an error without them. As a matter of taste, it seems like
a good idea though.
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 21:02 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-21 10:58 ` Andy Fan <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Andy Fan @ 2023-08-21 10:58 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
>
>
> Interestingly, when I relabel both places, like this:
>
> Oid targetOid = fexpr->funcresulttype;
> Const *target = makeConst(
> OIDOID, -1, InvalidOid, sizeof(Oid),
> ObjectIdGetDatum(targetOid), false, true);
> RelabelType *rTarget = makeRelabelType((Expr *)target,
> INTERNALOID, -1, InvalidOid, COERCE_IMPLICIT_CAST);
> fexpr->funcid = new_func_id;
> fexpr->args = opexpr->args;
> fexpr->args = list_insert_nth(fexpr->args, 0, rTarget);
> expr = (Expr *)makeRelabelType((Expr *)fexpr,
> targetOid, -1, InvalidOid, COERCE_IMPLICIT_CAST);
> }
> PG_RETURN_POINTER(expr);
>
> EXPLAIN looks like this:
>
> Seq Scan on pg_temp.test_jsonb
> Output: jsonb_array_element_type(('23'::oid)::internal, test_json,
> 0), (test_json -> 0)
> Filter: (test_jsonb.json_type = 'scalarint'::text)
>
> With COERCE_IMPLICIT_CAST both places, the relabeling of the
> function result is invisible, but the relabeling of the argument
> is visible.
>
>
I think this is because get_rule_expr's showimplicit is always
true for args in this case, checking the implementation of
get_rule_expr, I found PG behavior like this in many places.
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-21 01:31 ` Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-21 01:31 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Sat, Aug 19, 2023 at 3:09 AM Chapman Flack <[email protected]> wrote:
> On 2023-08-18 14:50, Chapman Flack wrote:
> > Now, my guess is EXPLAIN is complaining when it sees the Const
> > of type internal, and doesn't know how to show that value.
> > Perhaps makeRelabelType is the answer there, too: what if the
> > Const has Oid type, so EXPLAIN can show it, and what's inserted
> > as the function argument is a relabel node saying it's internal?
>
Simply changing the Const to be of type Oid makes EXPLAIN happy,
> and nothing ever says "hey, why are you passing this oid for an
> arg that wants internal?". This is without adding any relabel
> nodes anywhere.
>
Highlighting the user case of makeRelableType is interesting! But using
the Oid directly looks more promising for this question IMO, it looks like:
"you said we can put anything in this arg, so I put an OID const here",
seems nothing is wrong. Compared with the makeRelableType method,
I think the current method is more straightforward. Compared with
anyelement, it avoids the creation of makeDummyConst which I'm not
sure the implementation is alway correct. So I am pretty inclined to this
way!
v10 attached.
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v10-0001-optimize-casting-jsonb-to-a-given-type.patch (31.7K, 3-v10-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From c4b1ae13a0f4ba28972835ffa4c9850e2e0dbda6 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Wed, 16 Aug 2023 14:04:27 +0800
Subject: [PATCH v10 1/2] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/nodes/makefuncs.c | 30 +++++
src/backend/utils/adt/jsonb.c | 177 +++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 115 +++++++++++-------
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 32 +++--
src/include/nodes/makefuncs.h | 2 +
src/include/utils/jsonb.h | 1 +
src/test/regress/expected/jsonb.out | 178 +++++++++++++++++-----------
src/test/regress/sql/jsonb.sql | 56 ++++++---
9 files changed, 462 insertions(+), 131 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 0e7e6e46d94..9cb9178f01a 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -20,6 +20,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "utils/errcodes.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
@@ -352,6 +353,35 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
+/*
+ * makeDummyConst
+ * create a Const node with the specified type/typmod.
+ *
+ * This is a convenience routine to create a Const which only the
+ * type is interesting but make sure the value is accessible.
+ */
+Const *
+makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
+{
+ int16 typLen;
+ bool typByVal;
+ Const *c;
+ Datum val = 0;
+
+
+ get_typlenbyval(consttype, &typLen, &typByVal);
+
+ if (consttype == NUMERICOID)
+ val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
+ else if (!typByVal)
+ elog(ERROR, "create dummy const for type %u is not supported.", consttype);
+
+ /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
+ c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
+
+ return c;
+}
+
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..148c1e2e195 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,14 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2041,180 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc(sizeof(FuncExpr));
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_TYPE;
+
+ if (OidIsValid(new_func_id))
+ {
+ Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ fexpr->funcid = new_func_id;
+ fexpr->args = opexpr->args;
+ fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ }
+
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
+{
+ switch(targetOid)
+ {
+ Datum retValue;
+
+ case BOOLOID:
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "bool");
+ PG_RETURN_BOOL(v->val.boolean);
+
+ case NUMERICOID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "numeric");
+ PG_RETURN_NUMERIC(v->val.numeric);
+ case INT2OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "smallint");
+ retValue = DirectFunctionCall1(numeric_int2,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+ case INT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "integer");
+ retValue = DirectFunctionCall1(numeric_int4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case INT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "bigint");
+ retValue = DirectFunctionCall1(numeric_int8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "real");
+ retValue = DirectFunctionCall1(numeric_float4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "double precision");
+ retValue = DirectFunctionCall1(numeric_float8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ default:
+ elog(ERROR, "cast jsonb to type %u is not allowed", targetOid);
+ break;
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+jsonb_object_field_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ text *key = PG_GETARG_TEXT_PP(2);
+
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
+Datum
+jsonb_array_element_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ int element = PG_GETARG_INT32(2);
+
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..bb4ca807d74 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,40 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1551,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1605,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1626,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1638,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1663,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index f507b49bb28..8a896f9aad2 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202307261
+#define CATALOG_VERSION_NO 202308171
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989a..b6844537529 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4575,25 +4575,26 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prosupport => 'jsonb_cast_support',
+ prorettype => 'numeric', proargtypes => 'jsonb',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9928,6 +9929,13 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
+ proname => 'jsonb_object_field_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ prosrc => 'jsonb_object_field_type'},
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -9936,6 +9944,10 @@
proname => 'jsonb_array_element_text', prorettype => 'text',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
+{ oid => '4549', descr => 'cast an array element to given type',
+ proname => 'jsonb_array_element_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
proargtypes => 'jsonb _text', proallargtypes => '{jsonb,_text}',
@@ -9947,6 +9959,12 @@
proallargtypes => '{jsonb,_text}', proargmodes => '{i,v}',
proargnames => '{from_json,path_elems}',
prosrc => 'jsonb_extract_path_text' },
+{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
+ proname => 'jsonb_extract_path_type', provariadic => 'text',
+ prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
+ proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ proargnames => '{target_type,from_json,path_elems}',
+ prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
proname => 'jsonb_array_elements', prorows => '100', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 31807030055..cfbe5b26196 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,6 +58,8 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
+extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
+
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index 649a1644f24..532225314a9 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -435,5 +435,6 @@ extern Datum jsonb_build_object_worker(int nargs, Datum *args, bool *nulls,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null);
+extern Datum cast_jsonbvalue_to_type(JsonbValue *v, Oid target_oid);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..12daacb3b80 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,6 +457,7 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
@@ -501,10 +502,25 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
val2
(1 row)
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+ ?column? | ?column?
+----------+------------
+ | "a scalar"
+(1 row)
+
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+-----------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
?column?
----------
-
+ 2
(1 row)
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
@@ -1786,6 +1802,12 @@ select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
{"a": {"b": {"c": "foo"}}}
(1 row)
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
+ int2 | ?column? | int2
+------+----------+------
+ 2 | |
+(1 row)
+
select '[1,2,3]'::jsonb #> '{}';
?column?
-----------
@@ -3537,6 +3559,24 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.testjsonb
+ Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+(2 rows)
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
jsonb
@@ -5471,107 +5511,113 @@ select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
(1 row)
-- casts
-select 'true'::jsonb::bool;
- bool
-------
- t
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
+ bool | bool
+------+------
+ t | t
(1 row)
select '[]'::jsonb::bool;
ERROR: cannot cast jsonb array to type boolean
-select '1.0'::jsonb::float;
- float8
---------
- 1
+select ('{"a": []}'::jsonb->'a')::bool;
+ERROR: cannot cast jsonb array to type boolean
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
+ float8 | float8
+--------+--------
+ 1 | 1
(1 row)
select '[1.0]'::jsonb::float;
ERROR: cannot cast jsonb array to type double precision
-select '12345'::jsonb::int4;
- int4
--------
- 12345
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+ERROR: cannot cast jsonb array to type double precision
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
select '"hello"'::jsonb::int4;
ERROR: cannot cast jsonb string to type integer
-select '12345'::jsonb::numeric;
- numeric
----------
- 12345
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+ERROR: cannot cast jsonb string to type integer
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
+ numeric | numeric
+---------+---------
+ 12345 | 12345
(1 row)
select '{}'::jsonb::numeric;
ERROR: cannot cast jsonb object to type numeric
-select '12345.05'::jsonb::numeric;
- numeric
-----------
- 12345.05
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+ numeric | numeric
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float4;
- float4
-----------
- 12345.05
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+ float4 | float4
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float8;
- float8
-----------
- 12345.05
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+ float8 | float8
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::int2;
- int2
--------
- 12345
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int4;
- int4
--------
- 12345
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int8;
- int8
--------
- 12345
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
- numeric
-------------------------------------------------------
- 12345.0000000000000000000000000000000000000000000005
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+ numeric | numeric
+------------------------------------------------------+------------------------------------------------------
+ 12345.0000000000000000000000000000000000000000000005 | 12345.0000000000000000000000000000000000000000000005
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
- float4
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+ float4 | float4
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
- float8
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+ float8 | float8
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
- int2
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
- int4
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
- int8
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..8634d154efe 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,6 +154,7 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
@@ -166,7 +167,10 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'array';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 9 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'object';
@@ -491,6 +495,7 @@ SELECT '{"f2":["f3",1],"f4":{"f5":99,"f6":"stringy"}}'::jsonb#>>array['f2','1'];
-- corner cases for same
select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
select '[1,2,3]'::jsonb #> '{}';
select '"foo"'::jsonb #> '{}';
select '42'::jsonb #> '{}';
@@ -939,6 +944,19 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
SELECT '{"ff":{"a":12,"b":16},"qq":123}'::jsonb;
@@ -1496,23 +1514,27 @@ select ts_headline('{}'::jsonb, tsquery('aaa & bbb'));
select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
-- casts
-select 'true'::jsonb::bool;
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
select '[]'::jsonb::bool;
-select '1.0'::jsonb::float;
+select ('{"a": []}'::jsonb->'a')::bool;
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
select '[1.0]'::jsonb::float;
-select '12345'::jsonb::int4;
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
select '"hello"'::jsonb::int4;
-select '12345'::jsonb::numeric;
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
select '{}'::jsonb::numeric;
-select '12345.05'::jsonb::numeric;
-select '12345.05'::jsonb::float4;
-select '12345.05'::jsonb::float8;
-select '12345.05'::jsonb::int2;
-select '12345.05'::jsonb::int4;
-select '12345.05'::jsonb::int8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
--
2.21.0
[application/octet-stream] v10-0002-convert-anyelement-to-internal.patch (11.0K, 4-v10-0002-convert-anyelement-to-internal.patch)
download | inline diff:
From 84e268108100461f08ca6c909139777d50f14582 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Fri, 18 Aug 2023 15:38:50 +0800
Subject: [PATCH v10 2/2] convert anyelement to internal.
---
src/backend/nodes/makefuncs.c | 29 -----------------------------
src/backend/utils/adt/jsonb.c | 10 ++++++----
src/backend/utils/adt/jsonfuncs.c | 2 +-
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 12 ++++++------
src/include/nodes/makefuncs.h | 2 --
src/test/regress/expected/jsonb.out | 12 ++++++------
7 files changed, 20 insertions(+), 49 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 9cb9178f01a..a41fdddc662 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -353,35 +353,6 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
-/*
- * makeDummyConst
- * create a Const node with the specified type/typmod.
- *
- * This is a convenience routine to create a Const which only the
- * type is interesting but make sure the value is accessible.
- */
-Const *
-makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
-{
- int16 typLen;
- bool typByVal;
- Const *c;
- Datum val = 0;
-
-
- get_typlenbyval(consttype, &typLen, &typByVal);
-
- if (consttype == NUMERICOID)
- val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
- else if (!typByVal)
- elog(ERROR, "create dummy const for type %u is not supported.", consttype);
-
- /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
- c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
-
- return c;
-}
-
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 148c1e2e195..d893e9c14b5 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -2091,10 +2091,12 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
if (OidIsValid(new_func_id))
{
- Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ Const * target_typ = makeConst(OIDOID, -1, InvalidOid, sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false, true);
fexpr->funcid = new_func_id;
fexpr->args = opexpr->args;
- fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ fexpr->args = list_insert_nth(fexpr->args, 0, (void *) target_typ);
}
PG_RETURN_POINTER(fexpr);
@@ -2164,7 +2166,7 @@ cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
Datum
jsonb_object_field_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
text *key = PG_GETARG_TEXT_PP(2);
@@ -2188,7 +2190,7 @@ jsonb_object_field_type(PG_FUNCTION_ARGS)
Datum
jsonb_array_element_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
int element = PG_GETARG_INT32(2);
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index bb4ca807d74..02db9f53b47 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -1477,7 +1477,7 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 8a896f9aad2..8a919e1178b 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308171
+#define CATALOG_VERSION_NO 202308211
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b6844537529..66d1af71586 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9930,8 +9930,8 @@
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
- proname => 'jsonb_object_field_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ proname => 'jsonb_object_field_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb text', proargnames => '{target_type, from_json, field_name}',
prosrc => 'jsonb_object_field_type'},
{ oid => '3814', descr => 'planner support for numeric(jsonb)',
proname => 'jsonb_cast_support', prorettype => 'internal',
@@ -9945,8 +9945,8 @@
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
{ oid => '4549', descr => 'cast an array element to given type',
- proname => 'jsonb_array_element_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ proname => 'jsonb_array_element_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb int4', proargnames => '{target_type, from_json, element_index}',
prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
@@ -9961,8 +9961,8 @@
prosrc => 'jsonb_extract_path_text' },
{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
proname => 'jsonb_extract_path_type', provariadic => 'text',
- prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
- proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ prorettype => 'internal', proargtypes => 'internal jsonb _text',
+ proallargtypes => '{internal,jsonb,_text}', proargmodes => '{i,i,v}',
proargnames => '{target_type,from_json,path_elems}',
prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index cfbe5b26196..31807030055 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,8 +58,6 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
-extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
-
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 12daacb3b80..8ed80a11176 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -510,10 +510,10 @@ SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar'
explain (verbose, costs off)
SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
- QUERY PLAN
------------------------------------------------------------------------
+ QUERY PLAN
+------------------------------------------------------------------------------------------
Seq Scan on pg_temp.test_jsonb
- Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Output: pg_catalog.jsonb_array_element_type('23'::oid, test_json, 0), (test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
(3 rows)
@@ -3571,10 +3571,10 @@ SELECT (j->'a')::numeric,
(j #> '{"a"}')::numeric,
(j->0)::numeric
FROM testjsonb;
- QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Seq Scan on public.testjsonb
- Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+ Output: pg_catalog.jsonb_object_field_type('1700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('21'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('23'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('20'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('701'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('16'::oid, j, 'a'::text), pg_catalog.jsonb_extract_path_type('1700'::oid, j, '{a}'::text[]), pg_catalog.jsonb_array_element_type('1700'::oid, j, 0)
(2 rows)
-- nested tests
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-21 03:19 ` Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-21 03:19 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-20 21:31, Andy Fan wrote:
> Highlighting the user case of makeRelableType is interesting! But using
> the Oid directly looks more promising for this question IMO, it looks
> like:
> "you said we can put anything in this arg, so I put an OID const
> here",
> seems nothing is wrong.
Perhaps one of the more senior developers will chime in, but to me,
leaving out the relabel nodes looks more like "all of PostgreSQL's
type checking happened before the SupportRequestSimplify, so nothing
has noticed that we rewrote the tree with mismatched types, and as
long as nothing crashes we sort of got away with it."
Suppose somebody writes an extension to double-check that plan
trees are correctly typed. Or improves EXPLAIN to check a little more
carefully than it seems to. Omitting the relabel nodes could spell
trouble then.
Or, someone more familiar with the code than I am might say "oh,
mismatches like that are common in rewritten trees, we live with it."
But unless somebody tells me that, I'm not believing it.
But I would say we have proved the concept of SupportRequestSimplify
for this task. :)
Now, it would make me happy to further reduce some of the code
duplication between the original and the _type versions of these
functions. I see that you noticed the duplication in the case of
jsonb_extract_path, and you factored out jsonb_get_jsonbvalue so
it could be reused. There is also some duplication with object_field
and array_element. (Also, we may have overlooked jsonb_path_query
and jsonb_path_query_first as candidates for the source of the
cast. Two more candidates; five total.)
Here is one way this could be structured. Observe that every one
of those five source candidates operates in two stages:
Start: All of the processing until a JsonbValue has been obtained.
Finish: Converting the JsonbValue to some form for return.
Before this patch, there were two choices for Finish:
JsonbValueToJsonb or JsonbValueAsText.
With this patch, there are four Finish choices: those two, plus
PG_RETURN_BOOL(v->val.boolean), PG_RETURN_NUMERIC(v->val.numeric).
Clearly, with rewriting, we can avoid 5✕4 = 20 distinct
functions. The five candidate functions only differ in Start.
Suppose each of those had a _start version, like
jsonb_object_field_start, that only proceeds as far as
obtaining the JsonbValue, and returns that directly (an
'internal' return type). Naturally, each _start function would
need an 'internal' parameter also, even if it isn't used,
just to make sure it is not SQL-callable.
Now consider four Finish functions: jsonb_finish_jsonb,
jsonb_finish_text, jsonb_finish_boolean, jsonb_finish_numeric.
Each would have one 'internal' parameter (a JsonbValue), and
its return type declared normally. There is no need to pass
a type oid to any of these, and they need not contain any
switch to select a return type. The correct finisher to use
is simply chosen once at the time of rewriting.
So cast(jsonb_array_element(jb, 0) as numeric) would just get
rewritten as jsonb_finish_numeric(jsonb_array_element_start(jb,0)).
The other (int and float) types don't need new code; just have
the rewriter add a cast-from-numeric node on top. That's all
those other switch cases in cast_jsonbvalue_to_type are doing,
anyway.
Notice in this structure, less relabeling is needed. The
final return does not need relabeling, because each finish
function has the expected return type. Each finish function's
parameter is typed 'internal' (a JsonbValue), but that's just
what each start function returns, so no relabeling needed
there either.
The rewriter will have to supply some 'internal' constant
as a start-function parameter (because of the necessary
'internal' parameter). It might still be civilized to relabel
that.
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-22 03:14 ` Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-22 03:14 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
（Just relalized this was sent to chap in private, resent it again).
On Mon, Aug 21, 2023 at 6:50 PM Andy Fan <[email protected]> wrote:
>
>
> On Mon, Aug 21, 2023 at 11:19 AM Chapman Flack <[email protected]>
> wrote:
>
>> On 2023-08-20 21:31, Andy Fan wrote:
>> > Highlighting the user case of makeRelableType is interesting! But using
>> > the Oid directly looks more promising for this question IMO, it looks
>> > like:
>> > "you said we can put anything in this arg, so I put an OID const
>> > here",
>> > seems nothing is wrong.
>>
>> Perhaps one of the more senior developers will chime in, but to me,
>> leaving out the relabel nodes looks more like "all of PostgreSQL's
>> type checking happened before the SupportRequestSimplify, so nothing
>> has noticed that we rewrote the tree with mismatched types, and as
>> long as nothing crashes we sort of got away with it."
>>
>> Suppose somebody writes an extension to double-check that plan
>> trees are correctly typed. Or improves EXPLAIN to check a little more
>> carefully than it seems to. Omitting the relabel nodes could spell
>> trouble then.
>>
>> Or, someone more familiar with the code than I am might say "oh,
>> mismatches like that are common in rewritten trees, we live with it."
>> But unless somebody tells me that, I'm not believing it.
>>
>
> Well, this sounds long-lived. I kind of prefer to label it now. Adding
> the 3rd commit to relabel the arg and return value.
>
>
>> But I would say we have proved the concept of SupportRequestSimplify
>> for this task. :)
>>
>
> Yes, this is great!
>
>
>> Now, it would make me happy to further reduce some of the code
>> duplication between the original and the _type versions of these
>> functions. I see that you noticed the duplication in the case of
>> jsonb_extract_path, and you factored out jsonb_get_jsonbvalue so
>> it could be reused. There is also some duplication with object_field
>> and array_element.
>
>
Yes, compared with jsonb_extract_path, object_field and array_element
just have much less duplication, which are 2 lines and 6 lines separately.
> (Also, we may have overlooked jsonb_path_query
>> and jsonb_path_query_first as candidates for the source of the
>> cast. Two more candidates; five total.)
>>
>
I can try to add them at the same time when we talk about the
infrastruct, thanks for the hint!
>> Here is one way this could be structured. Observe that every one
>> of those five source candidates operates in two stages:
>>
>
> I'm not very excited with this manner, reasons are: a). It will have
> to emit more steps in ExprState->steps which will be harmful for
> execution. The overhead is something I'm not willing to afford.
> b). this manner requires more *internal*, which is kind of similar
> to "void *" in C. Could you explain more about the benefits of this?
>
> --
> Best Regards
> Andy Fan
>
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v10-0003-relabel-the-arg-and-resultvalue-with-INTERNALOID.patch (6.1K, 3-v10-0003-relabel-the-arg-and-resultvalue-with-INTERNALOID.patch)
download | inline diff:
From b35153f8ddb8d47bbb8ef5af62115f9f9287f309 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Mon, 21 Aug 2023 18:37:31 +0800
Subject: [PATCH v10 3/3] relabel the arg and resultvalue with INTERNALOID.
---
src/backend/utils/adt/jsonb.c | 12 ++++++++++--
src/test/regress/expected/jsonb.out | 12 ++++++------
2 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index d893e9c14b5..87350c6c912 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -2091,12 +2091,20 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
if (OidIsValid(new_func_id))
{
- Const * target_typ = makeConst(OIDOID, -1, InvalidOid, sizeof(Oid),
+ Const *target_typ = makeConst(OIDOID, -1, InvalidOid, sizeof(Oid),
ObjectIdGetDatum(fexpr->funcresulttype),
false, true);
+ /* Let others knows I'm an internal. */
+ RelabelType *rTarget = makeRelabelType((Expr *)target_typ,
+ INTERNALOID, -1,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
fexpr->funcid = new_func_id;
fexpr->args = opexpr->args;
- fexpr->args = list_insert_nth(fexpr->args, 0, (void *) target_typ);
+ fexpr->args = list_insert_nth(fexpr->args, 0, (void *) rTarget);
+
+ fexpr = (FuncExpr *)makeRelabelType((Expr *) fexpr, INTERNALOID,
+ 0, InvalidOid, COERCE_IMPLICIT_CAST);
}
PG_RETURN_POINTER(fexpr);
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 8ed80a11176..ad1af16bb3b 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -510,10 +510,10 @@ SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar'
explain (verbose, costs off)
SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
- QUERY PLAN
-------------------------------------------------------------------------------------------
+ QUERY PLAN
+-------------------------------------------------------------------------------------------
Seq Scan on pg_temp.test_jsonb
- Output: pg_catalog.jsonb_array_element_type('23'::oid, test_json, 0), (test_json -> 0)
+ Output: jsonb_array_element_type(('23'::oid)::internal, test_json, 0), (test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
(3 rows)
@@ -3571,10 +3571,10 @@ SELECT (j->'a')::numeric,
(j #> '{"a"}')::numeric,
(j->0)::numeric
FROM testjsonb;
- QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Seq Scan on public.testjsonb
- Output: pg_catalog.jsonb_object_field_type('1700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('21'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('23'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('20'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('701'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('16'::oid, j, 'a'::text), pg_catalog.jsonb_extract_path_type('1700'::oid, j, '{a}'::text[]), pg_catalog.jsonb_array_element_type('1700'::oid, j, 0)
+ Output: jsonb_object_field_type(('1700'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('21'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('23'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('20'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('700'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('701'::oid)::internal, j, 'a'::text), jsonb_object_field_type(('16'::oid)::internal, j, 'a'::text), pg_catalog.jsonb_extract_path_type(('1700'::oid)::internal, j, '{a}'::text[]), jsonb_array_element_type(('1700'::oid)::internal, j, 0)
(2 rows)
-- nested tests
--
2.21.0
[application/octet-stream] v10-0001-optimize-casting-jsonb-to-a-given-type.patch (31.7K, 4-v10-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From c4b1ae13a0f4ba28972835ffa4c9850e2e0dbda6 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Wed, 16 Aug 2023 14:04:27 +0800
Subject: [PATCH v10 1/3] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/nodes/makefuncs.c | 30 +++++
src/backend/utils/adt/jsonb.c | 177 +++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 115 +++++++++++-------
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 32 +++--
src/include/nodes/makefuncs.h | 2 +
src/include/utils/jsonb.h | 1 +
src/test/regress/expected/jsonb.out | 178 +++++++++++++++++-----------
src/test/regress/sql/jsonb.sql | 56 ++++++---
9 files changed, 462 insertions(+), 131 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 0e7e6e46d94..9cb9178f01a 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -20,6 +20,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "utils/errcodes.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
@@ -352,6 +353,35 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
+/*
+ * makeDummyConst
+ * create a Const node with the specified type/typmod.
+ *
+ * This is a convenience routine to create a Const which only the
+ * type is interesting but make sure the value is accessible.
+ */
+Const *
+makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
+{
+ int16 typLen;
+ bool typByVal;
+ Const *c;
+ Datum val = 0;
+
+
+ get_typlenbyval(consttype, &typLen, &typByVal);
+
+ if (consttype == NUMERICOID)
+ val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
+ else if (!typByVal)
+ elog(ERROR, "create dummy const for type %u is not supported.", consttype);
+
+ /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
+ c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
+
+ return c;
+}
+
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..148c1e2e195 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,14 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2041,180 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc(sizeof(FuncExpr));
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_TYPE;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_TYPE;
+
+ if (OidIsValid(new_func_id))
+ {
+ Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ fexpr->funcid = new_func_id;
+ fexpr->args = opexpr->args;
+ fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ }
+
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
+{
+ switch(targetOid)
+ {
+ Datum retValue;
+
+ case BOOLOID:
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "bool");
+ PG_RETURN_BOOL(v->val.boolean);
+
+ case NUMERICOID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "numeric");
+ PG_RETURN_NUMERIC(v->val.numeric);
+ case INT2OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "smallint");
+ retValue = DirectFunctionCall1(numeric_int2,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+ case INT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "integer");
+ retValue = DirectFunctionCall1(numeric_int4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case INT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "bigint");
+ retValue = DirectFunctionCall1(numeric_int8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT4OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "real");
+ retValue = DirectFunctionCall1(numeric_float4,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ case FLOAT8OID:
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "double precision");
+ retValue = DirectFunctionCall1(numeric_float8,
+ NumericGetDatum(v->val.numeric));
+ PG_RETURN_DATUM(retValue);
+
+ default:
+ elog(ERROR, "cast jsonb to type %u is not allowed", targetOid);
+ break;
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+Datum
+jsonb_object_field_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ text *key = PG_GETARG_TEXT_PP(2);
+
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
+Datum
+jsonb_array_element_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ int element = PG_GETARG_INT32(2);
+
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..bb4ca807d74 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,40 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_type(PG_FUNCTION_ARGS)
+{
+ Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Jsonb *jb = PG_GETARG_JSONB_P(1);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ return cast_jsonbvalue_to_type(v, targetOid);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1551,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1605,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1626,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1638,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1663,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index f507b49bb28..8a896f9aad2 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202307261
+#define CATALOG_VERSION_NO 202308171
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989a..b6844537529 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4575,25 +4575,26 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prosupport => 'jsonb_cast_support',
+ prorettype => 'numeric', proargtypes => 'jsonb',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9928,6 +9929,13 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
+ proname => 'jsonb_object_field_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ prosrc => 'jsonb_object_field_type'},
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -9936,6 +9944,10 @@
proname => 'jsonb_array_element_text', prorettype => 'text',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
+{ oid => '4549', descr => 'cast an array element to given type',
+ proname => 'jsonb_array_element_type', prorettype => 'anyelement',
+ proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
proargtypes => 'jsonb _text', proallargtypes => '{jsonb,_text}',
@@ -9947,6 +9959,12 @@
proallargtypes => '{jsonb,_text}', proargmodes => '{i,v}',
proargnames => '{from_json,path_elems}',
prosrc => 'jsonb_extract_path_text' },
+{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
+ proname => 'jsonb_extract_path_type', provariadic => 'text',
+ prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
+ proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ proargnames => '{target_type,from_json,path_elems}',
+ prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
proname => 'jsonb_array_elements', prorows => '100', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 31807030055..cfbe5b26196 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,6 +58,8 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
+extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
+
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index 649a1644f24..532225314a9 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -435,5 +435,6 @@ extern Datum jsonb_build_object_worker(int nargs, Datum *args, bool *nulls,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null);
+extern Datum cast_jsonbvalue_to_type(JsonbValue *v, Oid target_oid);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..12daacb3b80 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,6 +457,7 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
@@ -501,10 +502,25 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
val2
(1 row)
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+ ?column? | ?column?
+----------+------------
+ | "a scalar"
+(1 row)
+
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+-----------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
?column?
----------
-
+ 2
(1 row)
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
@@ -1786,6 +1802,12 @@ select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
{"a": {"b": {"c": "foo"}}}
(1 row)
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
+ int2 | ?column? | int2
+------+----------+------
+ 2 | |
+(1 row)
+
select '[1,2,3]'::jsonb #> '{}';
?column?
-----------
@@ -3537,6 +3559,24 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.testjsonb
+ Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+(2 rows)
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
jsonb
@@ -5471,107 +5511,113 @@ select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
(1 row)
-- casts
-select 'true'::jsonb::bool;
- bool
-------
- t
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
+ bool | bool
+------+------
+ t | t
(1 row)
select '[]'::jsonb::bool;
ERROR: cannot cast jsonb array to type boolean
-select '1.0'::jsonb::float;
- float8
---------
- 1
+select ('{"a": []}'::jsonb->'a')::bool;
+ERROR: cannot cast jsonb array to type boolean
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
+ float8 | float8
+--------+--------
+ 1 | 1
(1 row)
select '[1.0]'::jsonb::float;
ERROR: cannot cast jsonb array to type double precision
-select '12345'::jsonb::int4;
- int4
--------
- 12345
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+ERROR: cannot cast jsonb array to type double precision
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
select '"hello"'::jsonb::int4;
ERROR: cannot cast jsonb string to type integer
-select '12345'::jsonb::numeric;
- numeric
----------
- 12345
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+ERROR: cannot cast jsonb string to type integer
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
+ numeric | numeric
+---------+---------
+ 12345 | 12345
(1 row)
select '{}'::jsonb::numeric;
ERROR: cannot cast jsonb object to type numeric
-select '12345.05'::jsonb::numeric;
- numeric
-----------
- 12345.05
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+ numeric | numeric
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float4;
- float4
-----------
- 12345.05
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+ float4 | float4
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::float8;
- float8
-----------
- 12345.05
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+ float8 | float8
+----------+----------
+ 12345.05 | 12345.05
(1 row)
-select '12345.05'::jsonb::int2;
- int2
--------
- 12345
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int4;
- int4
--------
- 12345
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.05'::jsonb::int8;
- int8
--------
- 12345
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
- numeric
-------------------------------------------------------
- 12345.0000000000000000000000000000000000000000000005
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+ numeric | numeric
+------------------------------------------------------+------------------------------------------------------
+ 12345.0000000000000000000000000000000000000000000005 | 12345.0000000000000000000000000000000000000000000005
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
- float4
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+ float4 | float4
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
- float8
---------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+ float8 | float8
+--------+--------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
- int2
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+ int2 | int2
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
- int4
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+ int4 | int4
+-------+-------
+ 12345 | 12345
(1 row)
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
- int8
--------
- 12345
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
+ int8 | int8
+-------+-------
+ 12345 | 12345
(1 row)
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..8634d154efe 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,6 +154,7 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
@@ -166,7 +167,10 @@ SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'array';
SELECT test_json ->> 'field2' FROM test_jsonb WHERE json_type = 'object';
-SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'scalar';
+SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar';
+explain (verbose, costs off)
+SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 9 FROM test_jsonb WHERE json_type = 'array';
SELECT test_json -> 2 FROM test_jsonb WHERE json_type = 'object';
@@ -491,6 +495,7 @@ SELECT '{"f2":["f3",1],"f4":{"f5":99,"f6":"stringy"}}'::jsonb#>>array['f2','1'];
-- corner cases for same
select '{"a": {"b":{"c": "foo"}}}'::jsonb #> '{}';
+select ('2'::jsonb #> '{}')::int2, ('{"a":2}'::jsonb #> '{"b"}'), ('{"a":2}'::jsonb #> '{"b"}')::int2;
select '[1,2,3]'::jsonb #> '{}';
select '"foo"'::jsonb #> '{}';
select '42'::jsonb #> '{}';
@@ -939,6 +944,19 @@ SELECT count(*) FROM testjsonb WHERE j @? '$.bar';
RESET enable_seqscan;
DROP INDEX jidx;
+-- test the supported function for jsonb cast.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (j->'a')::numeric,
+(j->'a')::int2,
+(j->'a')::int4,
+(j->'a')::int8,
+(j->'a')::float4,
+(j->'a')::float8,
+(j->'a')::bool,
+(j #> '{"a"}')::numeric,
+(j->0)::numeric
+FROM testjsonb;
+
-- nested tests
SELECT '{"ff":{"a":12,"b":16}}'::jsonb;
SELECT '{"ff":{"a":12,"b":16},"qq":123}'::jsonb;
@@ -1496,23 +1514,27 @@ select ts_headline('{}'::jsonb, tsquery('aaa & bbb'));
select ts_headline('[]'::jsonb, tsquery('aaa & bbb'));
-- casts
-select 'true'::jsonb::bool;
+select 'true'::jsonb::bool, ('{"a": true}'::jsonb->'a')::bool;
select '[]'::jsonb::bool;
-select '1.0'::jsonb::float;
+select ('{"a": []}'::jsonb->'a')::bool;
+select '1.0'::jsonb::float, ('{"a": 1.0}'::jsonb->'a')::float;
select '[1.0]'::jsonb::float;
-select '12345'::jsonb::int4;
+select ('{"a": [1.0]}'::jsonb->'a')::float;
+select '12345'::jsonb::int4, ('{"a": 12345}'::jsonb->'a')::int4;
select '"hello"'::jsonb::int4;
-select '12345'::jsonb::numeric;
+select ('{"a": "hello"}'::jsonb->'a')::int4;
+
+select '12345'::jsonb::numeric, ('{"a": 12345}'::jsonb->'a')::numeric;
select '{}'::jsonb::numeric;
-select '12345.05'::jsonb::numeric;
-select '12345.05'::jsonb::float4;
-select '12345.05'::jsonb::float8;
-select '12345.05'::jsonb::int2;
-select '12345.05'::jsonb::int4;
-select '12345.05'::jsonb::int8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4;
-select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8;
+select '12345.05'::jsonb::numeric, ('{"a": 12345.05}'::jsonb->'a')::numeric;
+select '12345.05'::jsonb::float4, ('{"a": 12345.05}'::jsonb->'a')::float4;
+select '12345.05'::jsonb::float8, ('{"a": 12345.05}'::jsonb->'a')::float8;
+select '12345.05'::jsonb::int2, ('{"a": 12345.05}'::jsonb->'a')::int2;
+select '12345.05'::jsonb::int4, ('{"a": 12345.05}'::jsonb->'a')::int4;
+select '12345.05'::jsonb::int8, ('{"a": 12345.05}'::jsonb->'a')::int8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::numeric, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::numeric;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::float8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::float8;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int2, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int2;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int4, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int4;
+select '12345.0000000000000000000000000000000000000000000005'::jsonb::int8, ('{"a": 12345.0000000000000000000000000000000000000000000005}'::jsonb->'a')::int8;
--
2.21.0
[application/octet-stream] v10-0002-convert-anyelement-to-internal.patch (11.0K, 5-v10-0002-convert-anyelement-to-internal.patch)
download | inline diff:
From 84e268108100461f08ca6c909139777d50f14582 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Fri, 18 Aug 2023 15:38:50 +0800
Subject: [PATCH v10 2/3] convert anyelement to internal.
---
src/backend/nodes/makefuncs.c | 29 -----------------------------
src/backend/utils/adt/jsonb.c | 10 ++++++----
src/backend/utils/adt/jsonfuncs.c | 2 +-
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 12 ++++++------
src/include/nodes/makefuncs.h | 2 --
src/test/regress/expected/jsonb.out | 12 ++++++------
7 files changed, 20 insertions(+), 49 deletions(-)
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 9cb9178f01a..a41fdddc662 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -353,35 +353,6 @@ makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid)
typByVal);
}
-/*
- * makeDummyConst
- * create a Const node with the specified type/typmod.
- *
- * This is a convenience routine to create a Const which only the
- * type is interesting but make sure the value is accessible.
- */
-Const *
-makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid)
-{
- int16 typLen;
- bool typByVal;
- Const *c;
- Datum val = 0;
-
-
- get_typlenbyval(consttype, &typLen, &typByVal);
-
- if (consttype == NUMERICOID)
- val = DirectFunctionCall1(numeric_in, CStringGetDatum("0"));
- else if (!typByVal)
- elog(ERROR, "create dummy const for type %u is not supported.", consttype);
-
- /* XXX: here I assume constvalue=0 is accessible for constbyval.*/
- c = makeConst(consttype, consttypmod, 0, (int) typLen, val, false, typByVal);
-
- return c;
-}
-
/*
* makeBoolConst -
* creates a Const node representing a boolean value (can be NULL too)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 148c1e2e195..d893e9c14b5 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -2091,10 +2091,12 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
if (OidIsValid(new_func_id))
{
- Const *target = makeDummyConst(fexpr->funcresulttype, 0, InvalidOid);
+ Const * target_typ = makeConst(OIDOID, -1, InvalidOid, sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false, true);
fexpr->funcid = new_func_id;
fexpr->args = opexpr->args;
- fexpr->args = list_insert_nth(fexpr->args, 0, target);
+ fexpr->args = list_insert_nth(fexpr->args, 0, (void *) target_typ);
}
PG_RETURN_POINTER(fexpr);
@@ -2164,7 +2166,7 @@ cast_jsonbvalue_to_type(JsonbValue *v, Oid targetOid)
Datum
jsonb_object_field_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
text *key = PG_GETARG_TEXT_PP(2);
@@ -2188,7 +2190,7 @@ jsonb_object_field_type(PG_FUNCTION_ARGS)
Datum
jsonb_array_element_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
int element = PG_GETARG_INT32(2);
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index bb4ca807d74..02db9f53b47 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -1477,7 +1477,7 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path_type(PG_FUNCTION_ARGS)
{
- Oid targetOid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+ Oid targetOid = PG_GETARG_OID(0);
Jsonb *jb = PG_GETARG_JSONB_P(1);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(2);
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 8a896f9aad2..8a919e1178b 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308171
+#define CATALOG_VERSION_NO 202308211
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b6844537529..66d1af71586 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9930,8 +9930,8 @@
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
{ oid => '3813', descr => 'return a given type specified in desired_type from jsonb field',
- proname => 'jsonb_object_field_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb text', proargnames => '{target_type, from_json, field_name}',
+ proname => 'jsonb_object_field_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb text', proargnames => '{target_type, from_json, field_name}',
prosrc => 'jsonb_object_field_type'},
{ oid => '3814', descr => 'planner support for numeric(jsonb)',
proname => 'jsonb_cast_support', prorettype => 'internal',
@@ -9945,8 +9945,8 @@
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
prosrc => 'jsonb_array_element_text' },
{ oid => '4549', descr => 'cast an array element to given type',
- proname => 'jsonb_array_element_type', prorettype => 'anyelement',
- proargtypes => 'anyelement jsonb int4', proargnames => '{target_type, from_json, element_index}',
+ proname => 'jsonb_array_element_type', prorettype => 'internal',
+ proargtypes => 'internal jsonb int4', proargnames => '{target_type, from_json, element_index}',
prosrc => 'jsonb_array_element_type' },
{ oid => '3217', descr => 'get value from jsonb with path elements',
proname => 'jsonb_extract_path', provariadic => 'text', prorettype => 'jsonb',
@@ -9961,8 +9961,8 @@
prosrc => 'jsonb_extract_path_text' },
{ oid => '4551', descr => 'cast value from jsonb as text with path elements to given type',
proname => 'jsonb_extract_path_type', provariadic => 'text',
- prorettype => 'anyelement', proargtypes => 'anyelement jsonb _text',
- proallargtypes => '{anyelement,jsonb,_text}', proargmodes => '{i,i,v}',
+ prorettype => 'internal', proargtypes => 'internal jsonb _text',
+ proallargtypes => '{internal,jsonb,_text}', proargmodes => '{i,i,v}',
proargnames => '{target_type,from_json,path_elems}',
prosrc => 'jsonb_extract_path_type' },
{ oid => '3219', descr => 'elements of a jsonb array',
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index cfbe5b26196..31807030055 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -58,8 +58,6 @@ extern Const *makeConst(Oid consttype,
extern Const *makeNullConst(Oid consttype, int32 consttypmod, Oid constcollid);
-extern Const *makeDummyConst(Oid consttype, int32 consttypmod, Oid constcollid);
-
extern Node *makeBoolConst(bool value, bool isnull);
extern Expr *makeBoolExpr(BoolExprType boolop, List *args, int location);
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 12daacb3b80..8ed80a11176 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -510,10 +510,10 @@ SELECT test_json -> 2, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalar'
explain (verbose, costs off)
SELECT (test_json -> 0)::int4, test_json -> 0 FROM test_jsonb WHERE json_type = 'scalarint';
- QUERY PLAN
------------------------------------------------------------------------
+ QUERY PLAN
+------------------------------------------------------------------------------------------
Seq Scan on pg_temp.test_jsonb
- Output: jsonb_array_element_type(0, test_json, 0), (test_json -> 0)
+ Output: pg_catalog.jsonb_array_element_type('23'::oid, test_json, 0), (test_json -> 0)
Filter: (test_jsonb.json_type = 'scalarint'::text)
(3 rows)
@@ -3571,10 +3571,10 @@ SELECT (j->'a')::numeric,
(j #> '{"a"}')::numeric,
(j->0)::numeric
FROM testjsonb;
- QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Seq Scan on public.testjsonb
- Output: jsonb_object_field_type('0'::numeric, j, 'a'::text), jsonb_object_field_type('0'::smallint, j, 'a'::text), jsonb_object_field_type(0, j, 'a'::text), jsonb_object_field_type('0'::bigint, j, 'a'::text), jsonb_object_field_type('0'::real, j, 'a'::text), jsonb_object_field_type('0'::double precision, j, 'a'::text), jsonb_object_field_type(false, j, 'a'::text), pg_catalog.jsonb_extract_path_type('0'::numeric, j, '{a}'::text[]), jsonb_array_element_type('0'::numeric, j, 0)
+ Output: pg_catalog.jsonb_object_field_type('1700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('21'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('23'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('20'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('700'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('701'::oid, j, 'a'::text), pg_catalog.jsonb_object_field_type('16'::oid, j, 'a'::text), pg_catalog.jsonb_extract_path_type('1700'::oid, j, '{a}'::text[]), pg_catalog.jsonb_array_element_type('1700'::oid, j, 0)
(2 rows)
-- nested tests
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-22 05:54 ` Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-22 05:54 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
>
>
>>> Perhaps one of the more senior developers will chime in, but to me,
>>> leaving out the relabel nodes looks more like "all of PostgreSQL's
>>> type checking happened before the SupportRequestSimplify, so nothing
>>> has noticed that we rewrote the tree with mismatched types, and as
>>> long as nothing crashes we sort of got away with it."
>>>
>>> Suppose somebody writes an extension to double-check that plan
>>> trees are correctly typed. Or improves EXPLAIN to check a little more
>>> carefully than it seems to. Omitting the relabel nodes could spell
>>> trouble then.
>>>
>>> Or, someone more familiar with the code than I am might say "oh,
>>> mismatches like that are common in rewritten trees, we live with it."
>>> But unless somebody tells me that, I'm not believing it.
>>>
>>
>> Well, this sounds long-lived. I kind of prefer to label it now. Adding
>> the 3rd commit to relabel the arg and return value.
>>
>>
After we label it, we will get error like this:
select (a->'a')::int4 from m;
ERROR: cannot display a value of type internal
However the following statement can work well.
select ('{"a": 12345}'::jsonb->'a')::numeric;
numeric
---------
12345
That's mainly because the later query doesn't go through the planner
support function. I didn't realize this before so the test case doesn't
catch it. Will add the test case in the next version. The reason why
we get the error for the first query is because the query tree says
we should output an "internal" result at last and then pg doesn't
know how to output an internal data type. This is kind of in conflict
with our goal.
So currently the only choices are: PATCH 001 or PATCH 001 + 002.
https://www.postgresql.org/message-id/CAKU4AWrs4Pzajm2_tgtUTf%3DCWfDJEx%3D3h45Lhqg7tNOVZw5YxA%40mail...
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-22 12:16 ` Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-22 12:16 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-22 01:54, Andy Fan wrote:
> After we label it, we will get error like this:
>
> select (a->'a')::int4 from m;
> ERROR: cannot display a value of type internal
Without looking in depth right now, I would double-check
what relabel node is being applied at the result. The idea,
of course, was to relabel the result as the expected result
type, not internal.
(Or, as in the restructuring suggested earlier, to use a
finish function whose return type is already as expected,
and needs no relabeling.)
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-26 22:28 ` Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-26 22:28 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-22 08:16, Chapman Flack wrote:
> On 2023-08-22 01:54, Andy Fan wrote:
>> After we label it, we will get error like this:
>>
>> select (a->'a')::int4 from m;
>> ERROR: cannot display a value of type internal
>
> Without looking in depth right now, I would double-check
> what relabel node is being applied at the result. The idea,
> of course, was to relabel the result as the expected result
as I suspected, looking at v10-0003, there's this:
+ fexpr = (FuncExpr *)makeRelabelType((Expr *) fexpr, INTERNALOID,
+ 0, InvalidOid,
COERCE_IMPLICIT_CAST);
compared to the example I had sent earlier:
On 2023-08-18 17:02, Chapman Flack wrote:
> expr = (Expr *)makeRelabelType((Expr *)fexpr,
> targetOid, -1, InvalidOid, COERCE_IMPLICIT_CAST);
The key difference: this is the label going on the result type
of the function we are swapping in. The function already has
return type declared internal; we want to relabel it as
returning the type identified by targetOid. A relabel node
to type internal is the reverse of what's needed (and also
superfluous, as the function's return type is internal already).
Two more minor differences: (1) the node you get from
makeRelabelType is an Expr, but not really a FuncExpr. Casting
it to FuncExpr is a bit bogus. Also, the third argument to
makeRelabelType is a typmod, and I believe the "not-modified"
typmod is -1, not 0.
In the example I had sent earlier, there were two relabel nodes,
serving different purposes. In one, we have a function that wants
internal for an argument, but we've made a Const with type OIDOID,
so we want to say "this is internal, the thing the function wants."
The situation is reversed at the return type of the function: the
function declares its return type internal, but the surrounding
query is expecting an int4 (or whatever targetOid identifies),
so any relabel node there needs to say it's an int4 (or whatever).
So, that approach involves two relabelings. In the one idea for
restructuring that I suggested earlier, that's reduced: the
..._int function produces a JsonbValue (typed internal) and
the selected ..._finish function expects that, so those types
match and no relabeling is called for. And with the correct
..._finish function selected at rewrite time, it already has
the same return type the surrounding query expects, so no
relabeling is called for there either.
However, simply to ensure the ..._int function cannot be
casually called, it needs an internal parameter, even an
unused one, and the rewriter must supply a value for that,
which may call for one relabel node.
On 2023-08-21 06:50, Andy Fan wrote:
> I'm not very excited with this manner, reasons are: a). It will have
> to emit more steps in ExprState->steps which will be harmful for
> execution. The overhead is something I'm not willing to afford.
I would be open to a performance comparison, but offhand I am not
sure whether the overhead of another step or two in an ExprState
is appreciably more than some of the overhead in the present patch,
such as the every-time-through fcinfo initialization buried in
DirectFunctionCall1 where you don't necessarily see it. I bet
the fcinfo in an ExprState step gets set up once, and just has
new argument values slammed into it each time through.
(Also, I know very little about how the JIT compiler is used in PG,
but I suspect that a step you bury inside your function is a step
it may not get to see.)
> b). this manner requires more *internal*, which is kind of similar
> to "void *" in C.
I'm not sure in what sense you mean "more". The present patch
has to deal with two places where some other type must be
relabeled internal or something internal relabeled to another
type. The approach I suggested does involve two families of
function, one returning internal (a JsonbValue) and one
expecting internal (a JsonbValue), where the rewriter would
compose one over the other, no relabeling needed. There's
also an internal parameter needed for whatever returns
internal, and that's just the protocol for how such things
are done. To me, that seems like a fairly principled use of
the type.
I would not underestimate the benefit of reducing the code
duplication and keeping the patch as clear as possible.
The key contributions of the patch are getting a numeric or
boolean efficiently out of the JSON operation. Getting from
numeric to int or float are things the system already does
well. A patch that focuses on what it contributes, and avoids
redoing things the system already can do--unless the duplication
can be shown to have a strong performance benefit--is easier to
review and probably to get integrated.
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-30 04:47 ` Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-30 04:47 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
(Sorry for leaving this discussion for such a long time, how times fly!)
On Sun, Aug 27, 2023 at 6:28 AM Chapman Flack <[email protected]> wrote:
> On 2023-08-22 08:16, Chapman Flack wrote:
> > On 2023-08-22 01:54, Andy Fan wrote:
> >> After we label it, we will get error like this:
> >>
> >> select (a->'a')::int4 from m;
> >> ERROR: cannot display a value of type internal
> >
> > Without looking in depth right now, I would double-check
> > what relabel node is being applied at the result. The idea,
> > of course, was to relabel the result as the expected result
>
> as I suspected, looking at v10-0003, there's this:
>
> + fexpr = (FuncExpr *)makeRelabelType((Expr *) fexpr, INTERNALOID,
> + 0, InvalidOid,
> COERCE_IMPLICIT_CAST);
>
> compared to the example I had sent earlier:
>
> On 2023-08-18 17:02, Chapman Flack wrote:
> > expr = (Expr *)makeRelabelType((Expr *)fexpr,
> > targetOid, -1, InvalidOid, COERCE_IMPLICIT_CAST);
>
> The key difference: this is the label going on the result type
> of the function we are swapping in.
I'm feeling we have some understanding gap in this area, let's
see what it is. Suppose the original query is:
numeric(jsonb_object_field(v_jsonb, text)) -> numeric.
without the patch 003, the rewritten query is:
jsonb_object_field_type(NUMERICOID, v_jsonb, text) -> NUMERIC.
However the declared type of jsonb_object_field_type is:
jsonb_object_field_type(internal, jsonb, text) -> internal.
So the situation is: a). We input a CONST(type=OIDOID, ..) for an
internal argument. b). We return a NUMERIC type which matches
the original query c). result type NUMERIC doesn't match the declared
type 'internal' d). it doesn't match the run-time type of internal
argument which is OID.
case a) is fixed by RelableType. case b) shouldn't be treat as an
issue. I thought you wanted to address the case c), and patch
003 tries to fix it, then the ERROR above. At last I realized case
c) isn't the one you want to fix. case d) shouldn't be requirement
at the first place IIUC.
So your new method is:
1. jsonb_{op}_start() -> internal (internal actually JsonbValue).
2. jsonb_finish_{type}(internal, ..) -> type. (internal is JsonbValue ).
This avoids the case a) at the very beginning. I'd like to provides
patches for both solutions for comparison. Any other benefits of
this method I am missing?
> Two more minor differences: (1) the node you get from
> makeRelabelType is an Expr, but not really a FuncExpr. Casting
> it to FuncExpr is a bit bogus. Also, the third argument to
> makeRelabelType is a typmod, and I believe the "not-modified"
> typmod is -1, not 0.
>
My fault, you are right.
>
> On 2023-08-21 06:50, Andy Fan wrote:
> > I'm not very excited with this manner, reasons are: a). It will have
> > to emit more steps in ExprState->steps which will be harmful for
> > execution. The overhead is something I'm not willing to afford.
>
> I would be open to a performance comparison, but offhand I am not
> sure whether the overhead of another step or two in an ExprState
> is appreciably more than some of the overhead in the present patch,
> such as the every-time-through fcinfo initialization buried in
> DirectFunctionCall1 where you don't necessarily see it. I bet
the fcinfo in an ExprState step gets set up once, and just has
> new argument values slammed into it each time through.
>
fcinfo initialization in DirectFunctionCall1 is an interesting point!
so I am persuaded the extra steps in ExprState may not be
worse than the current way due to the "every-time-through
fcinfo initialization" (in which case the memory is allocated
once in heap rather than every time in stack). I can do a
comparison at last to see if we can find some other interesting
findings.
> I would not underestimate the benefit of reducing the code
> duplication and keeping the patch as clear as possible.
> The key contributions of the patch are getting a numeric or
> boolean efficiently out of the JSON operation. Getting from
> numeric to int or float are things the system already does
> well.
True, reusing the casting system should be better than hard-code
the casting function manually. I'd apply this on both methods.
> A patch that focuses on what it contributes, and avoids
> redoing things the system already can do--unless the duplication
> can be shown to have a strong performance benefit--is easier to
> review and probably to get integrated.
>
Agreed.
At last, thanks for the great insights and patience!
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-08-30 13:47 ` Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-08-30 13:47 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-08-30 00:47, Andy Fan wrote:
> see what it is. Suppose the original query is:
>
> numeric(jsonb_object_field(v_jsonb, text)) -> numeric.
> ...
> However the declared type of jsonb_object_field_type is:
>
> jsonb_object_field_type(internal, jsonb, text) -> internal.
>
> So the situation is: b). We return a NUMERIC type which matches
> the original query ...
> case b) shouldn't be treat as an issue.
*We* may know we are returning a NUMERIC type which matches the
original query, but nothing else knows that. Anything that
examined the complete tree after our rewriting would see some
expression that wants a numeric type, but supplied with a
subexpression that returns internal. Without a relabel node
there to promise that we know this internal is really numeric,
any type checker would reject the tree.
The fact that it even works at all without a relabel node there
seems to indicate that all of PostgreSQL's type checking was
done before calling the support function, and that there is not
much sanity checking of what the support function returns,
which I guess is efficient, if a little scary. Seems like
writing a support function is a bit like trapeze performing
without a net.
> So your new method is:
> 1. jsonb_{op}_start() -> internal (internal actually JsonbValue).
> 2. jsonb_finish_{type}(internal, ..) -> type. (internal is JsonbValue
> ).
>
> This avoids the case a) at the very beginning. I'd like to provides
> patches for both solutions for comparison.
I think, unavoidably, there is still a case a) at the very beginning,
just because of the rule that if json_{op}_start is going to have an
internal return type, it needs to have at least one internal parameter
to prevent casual calls from SQL, even if that parameter is not used
for anything.
It would be ok to write in a Const for that parameter, just zero or
42 or anything besides null (in case the function is strict), but
again if the Const has type internal then EXPLAIN will be sad, so
it has to be some type that makes EXPLAIN cheerful, and relabeled
internal.
But with this approach there is no longer a type mismatch of the
end result.
> fcinfo initialization in DirectFunctionCall1 is an interesting point!
> so I am persuaded the extra steps in ExprState may not be
> worse than the current way due to the "every-time-through
> fcinfo initialization" (in which case the memory is allocated
> once in heap rather than every time in stack).
Stack allocation is super cheap, just by emitting the function
entry to reserve n+m bytes instead of just m, so it there's any
measurable cost to the DirectFunctionCall I would think it more
likely to be in the initialization after allocation ... but I
haven't looked at that code closely to see how much there is.
I just wanted to make the point that another step or two in
ExprState might not be a priori worse. We might be talking
about negligible effects in either direction.
> I can do a
> comparison at last to see if we can find some other interesting
> findings.
That would be the way to find out. I think I would still lean
toward the approach with less code duplication, unless there
is a strong timing benefit the other way.
> True, reusing the casting system should be better than hard-code
> the casting function manually. I'd apply this on both methods.
I noticed there is another patch registered in this CF: [1]
It adds new operations within jsonpath like .bigint .time
and so on.
I was wondering whether that work would be conflicting or
complementary with this. It looks to be complementary. The
operations being added there are within jsonpath evaluation.
Here we are working on faster ways to get those results out.
It does not seem that [1] will add any new choices in
JsonbValue. All of its (.bigint .integer .number) seem to
verify the requested form and then put the result as a
numeric in ->val.numeric. So that doesn't add any new
cases for this patch to handle. (Too bad, in a way: if that
other patch added ->val.bigint, this patch could add a case
to retrieve that value without going through the work of
making a numeric. But that would complicate other things
touching JsonbValue, and be a matter for that other patch.)
It may be expanding the choices for what we might one day
find in ->val.datetime though.
Regards,
-Chap
[1] https://commitfest.postgresql.org/44/4526/
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-08-31 09:10 ` Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-08-31 09:10 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hi Chap,
The v11 attached, mainly changes are:
1. use the jsonb_xx_start and jsonb_finish_numeric style.
2. improve the test case a bit.
It doesn't include:
1. the jsonb_finish_text function, since we have a operator ->> for text
already and the performance for it is OK and there is no cast entry for
jsonb to text.
2. the jsonb_finish_jsonb since I can't see a clear user case for now.
Rewriting jsonb_object_field with 2 DirectFunctionCall looks not pretty
reasonable as we paid 2 DirectFunctionCall overhead to reduce ~10 lines
code duplication.
An incompatible issue at error message level is found during test:
create table jb(a jsonb);
insert into jb select '{"a": "a"}'::jsonb;
select (a->'a')::int4 from jb;
master: ERROR: cannot cast jsonb string to type *integer*
patch: ERROR: cannot cast jsonb string to type *numeric*
That's mainly because we first extract the field to numeric and
then cast it to int4 and the error raised at the first step and it
doesn't know the final type. One way to fix it is adding a 2nd
argument for jsonb_finish_numeric for the real type, but
it looks weird and more suggestions on this would be good.
Performance comparison between v10 and v11.
create table tb (a jsonb);
insert into tb select '{"a": 1}'::jsonb from generate_series(1, 100000)i;
select 1 from tb where (a->'a')::int2 = 2; (pgbench 5 times)
v11: 16.273 ms
v10: 15.986 ms
master: 32.530ms
So I think the performance would not be an issue.
> I noticed there is another patch registered in this CF: [1]
> It adds new operations within jsonpath like .bigint .time
> and so on.
>
> I was wondering whether that work would be conflicting or
> complementary with this. It looks to be complementary. The
> operations being added there are within jsonpath evaluation.
> Here we are working on faster ways to get those results out.
>
> It does not seem that [1] will add any new choices in
> JsonbValue. All of its (.bigint .integer .number) seem to
> verify the requested form and then put the result as a
> numeric in ->val.numeric. So that doesn't add any new
> cases for this patch to handle. (Too bad, in a way: if that
> other patch added ->val.bigint, this patch could add a case
> to retrieve that value without going through the work of
> making a numeric. But that would complicate other things
> touching JsonbValue, and be a matter for that other patch.)
>
> It may be expanding the choices for what we might one day
> find in ->val.datetime though.
>
> Thanks for this information. I tried the jsonb_xx_start and
jsonb_finish_numeric style, and it looks like a good experience
and it may not make things too complicated even if the above
things happen IMO.
Any feedback is welcome.
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v11-0001-optimize-casting-jsonb-to-a-given-type.patch (23.7K, 3-v11-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From 33c9395574cfa0e8040e56661aabec40d5a8aa6b Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Thu, 31 Aug 2023 16:48:35 +0800
Subject: [PATCH v11] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/utils/adt/jsonb.c | 169 ++++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 114 ++++++++++++-------
src/include/catalog/catversion.h | 3 +-
src/include/catalog/pg_proc.dat | 38 +++++--
src/test/regress/expected/jsonb.out | 78 ++++++++++++-
src/test/regress/sql/jsonb.sql | 48 +++++++-
6 files changed, 398 insertions(+), 52 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..449cb4a0523 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2042,171 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc0(sizeof(FuncExpr));
+ FuncExpr *jsonb_start_func = NULL, *jsonb_finish_func = NULL, *final_func = NULL;
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(fexpr);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, opexpr->args,
+ opexpr->opcollid, opexpr->inputcollid,
+ COERCE_EXPLICIT_CALL);
+
+ /* relabel the first arguments as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, 0,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make1(jsonb_start_func), opexpr->opcollid,
+ opexpr->inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ final_func = (FuncExpr *)coerce_type(NULL, (Node *)jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), opexpr->opcollid,
+ opexpr->inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(fexpr);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ text *key = PG_GETARG_TEXT_PP(1);
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ int element = PG_GETARG_INT32(1);
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, "numeric");
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..f6042ea442c 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,39 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1550,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1604,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1625,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1637,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1662,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index ab9a7ac1f79..2ce85b42d8f 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,5 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308251
-
+#define CATALOG_VERSION_NO 202308311
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9805bc61180..583032707e4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4587,25 +4587,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9947,6 +9947,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '3813',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4549',
+ proname => 'jsonb_extract_path_start', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{jsonb,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal', proargnames => '{from_jsonvalue}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..c166eb003f1 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,81 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)), (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)))::smallint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)))::integer, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)))::bigint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)))::real, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text)))::double precision, jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 0)), jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 10)), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), '{field6,f1}'::text[])), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), '{field6,f2}'::text[])), jsonb_finish_bool(jsonb_extract_path_start((test_json)::internal(0), '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), '{}'::text[]))
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field1'::text)))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type numeric
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +659,8 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+(7 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..9df88866671 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,54 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true}');
+
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-01 03:09 ` Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-09-01 03:09 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> An incompatible issue at error message level is found during test:
> create table jb(a jsonb);
> insert into jb select '{"a": "a"}'::jsonb;
> select (a->'a')::int4 from jb;
>
> master: ERROR: cannot cast jsonb string to type *integer*
> patch: ERROR: cannot cast jsonb string to type *numeric*
>
> That's mainly because we first extract the field to numeric and
> then cast it to int4 and the error raised at the first step and it
> doesn't know the final type. One way to fix it is adding a 2nd
> argument for jsonb_finish_numeric for the real type, but
> it looks weird and more suggestions on this would be good.
>
>
v12 is attached to address the above issue, I added a new argument
named target_oid for jsonb_finish_numeric so that it can raise a
correct error message. I also fixed the issue reported by opr_sanity
in this version.
Chap, do you still think we should refactor the code for the previous
existing functions like jsonb_object_field for less code duplication
purpose? I think we should not do it because a). The code duplication
is just ~10 rows. b). If we do the refactor, we have to implement
two DirectFunctionCall1. Point b) is the key reason I am not willing
to do it. Or do I miss other important reasons?
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v12-0001-optimize-casting-jsonb-to-a-given-type.patch (24.9K, 3-v12-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From 03765aec9b27b82d005a38dd1fa81e61d4032658 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Thu, 31 Aug 2023 16:48:35 +0800
Subject: [PATCH v12] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/utils/adt/jsonb.c | 177 ++++++++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 114 +++++++++++-------
src/include/catalog/catversion.h | 3 +-
src/include/catalog/pg_proc.dat | 38 ++++--
src/test/regress/expected/jsonb.out | 78 +++++++++++-
src/test/regress/sql/jsonb.sql | 48 +++++++-
6 files changed, 406 insertions(+), 52 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..14c946501da 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2042,179 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+static bool
+jsonb_cast_is_optimized(Oid target_type)
+{
+ switch(target_type)
+ {
+ case NUMERICOID:
+ case BOOLOID:
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc0(sizeof(FuncExpr));
+ FuncExpr *jsonb_start_func = NULL, *jsonb_finish_func = NULL, *final_func = NULL;
+ OpExpr *opexpr;
+ Oid new_func_id = InvalidOid;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+
+ opexpr = (OpExpr *) linitial(fexpr->args);
+
+ if (!IsA(opexpr, OpExpr) ||
+ !jsonb_cast_is_optimized(fexpr->funcresulttype))
+ {
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+ }
+
+ if (opexpr->opfuncid == F_JSONB_OBJECT_FIELD)
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ else if (opexpr->opfuncid == F_JSONB_ARRAY_ELEMENT)
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ else if (opexpr->opfuncid == F_JSONB_EXTRACT_PATH)
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(fexpr);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, opexpr->args,
+ opexpr->opcollid, opexpr->inputcollid,
+ COERCE_EXPLICIT_CALL);
+
+ /* relabel the first arguments as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, 0,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make2(jsonb_start_func,
+ makeConst(OIDOID,
+ -1,
+ InvalidOid,
+ sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false,
+ true)),
+ opexpr->opcollid, opexpr->inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ final_func = (FuncExpr *)coerce_type(NULL, (Node *)jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), opexpr->opcollid,
+ opexpr->inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(fexpr);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ text *key = PG_GETARG_TEXT_PP(1);
+ JsonbValue *v;
+ JsonbValue vbuf;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ &vbuf);
+
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ int element = PG_GETARG_INT32(1);
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ Oid final_oid = PG_GETARG_OID(1);
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, format_type_be(final_oid));
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..f6042ea442c 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -1473,6 +1474,39 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1550,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1604,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1625,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1637,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1662,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index ab9a7ac1f79..2ce85b42d8f 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,5 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308251
-
+#define CATALOG_VERSION_NO 202308311
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9805bc61180..4fa3eea2d1a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4587,25 +4587,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9947,6 +9947,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552', descr => 'extract jsonbvalue from jsonb for the given field',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '3813', descr => 'extract josnbvalue from jsonb array for the given index',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4549', descr => 'extract jsonbvalue from jsonb for the given paths',
+ proname => 'jsonb_extract_path_start', provariadic => 'text', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{internal,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553', descr => 'convert a jsonbvalue to numeric',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal oid', proargnames => '{from_jsonvalue,target_oid}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554', descr => 'convert a jsonbvalue to boolean',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..c12f3d06f3c 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,81 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '1700'::oid), (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '23'::oid))::integer, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '20'::oid))::bigint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '700'::oid))::real, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '701'::oid))::double precision, jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 0), '1700'::oid), jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 10), '1700'::oid), jsonb_finish_numeric(pg_catalog.jsonb_extract_path_start((test_json)::internal(0), '{field6,f1}'::text[]), '1700'::oid), jsonb_finish_numeric(pg_catalog.jsonb_extract_path_start((test_json)::internal(0), '{field6,f2}'::text[]), '1700'::oid), jsonb_finish_bool(pg_catalog.jsonb_extract_path_start((test_json)::internal(0), '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(pg_catalog.jsonb_extract_path_start((test_json)::internal(0), '{}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field1'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type integer
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +659,8 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+(7 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..9df88866671 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,54 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true}');
+
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-02 01:25 ` jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: jian he @ 2023-09-02 01:25 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: Chapman Flack <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
I think the last patch failed. I am not 100% sure.
https://cirrus-ci.com/task/5464366154252288
says "Created 21 hours ago", I assume the latest patch.
the diff in Artifacts section. you can go to
testrun/build/testrun/regress/regress/regression.diffs
diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/jsonb.out
/tmp/cirrus-ci-build/build/testrun/regress/regress/results/jsonb.out
--- /tmp/cirrus-ci-build/src/test/regress/expected/jsonb.out
2023-09-01 03:34:43.585036700 +0000
+++ /tmp/cirrus-ci-build/build/testrun/regress/regress/results/jsonb.out
2023-09-01 03:39:05.800452844 +0000
@@ -528,7 +528,7 @@
(3 rows)
SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type
= 'object';
-ERROR: cannot cast jsonb string to type integer
+ERROR: unknown jsonb type: 1125096840
SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type
= 'object';
ERROR: cannot cast jsonb string to type boolean
\pset null ''
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
@ 2023-09-04 11:43 ` Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-09-04 11:43 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Chapman Flack <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hi Jian,
SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type
> = 'object';
> -ERROR: cannot cast jsonb string to type integer
> +ERROR: unknown jsonb type: 1125096840
>
Thanks for the report! The reason is I return the address of a local
variable.
jsonb_object_field_start(PG_FUNCTION_ARGS)
{
JsonbValue *v;
JsonbValue vbuf;
v = getKeyJsonValueFromContainer(&jb->root,
VARDATA_ANY(key),\
VARSIZE_ANY_EXHDR(key),
&vbuf);
PG_RETURN_POINTER(v);
}
Here the v points to vbuf which is a local variable in stack. I'm confused
that why it works on my local machine and also works in the most queries
in cfbot, the fix is below
v = getKeyJsonValueFromContainer(&jb->root,
VARDATA_ANY(key),\
VARSIZE_ANY_EXHDR(key),
NULL);
I will send an updated version soon.
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-04 14:35 ` Andy Fan <[email protected]>
2023-09-05 12:51 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Andy Fan @ 2023-09-04 14:35 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Chapman Flack <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hi,
v13 attached. Changes includes:
1. fix the bug Jian provides.
2. reduce more code duplication without DirectFunctionCall.
3. add the overlooked jsonb_path_query and jsonb_path_query_first as
candidates
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v13-0001-optimize-casting-jsonb-to-a-given-type.patch (34.9K, 3-v13-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From 9b1b4446d41b587e0dd635a7668b42e00ab4e79b Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Mon, 4 Sep 2023 22:29:18 +0800
Subject: [PATCH v13] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/utils/adt/jsonb.c | 145 +++++++++++++++++++++++
src/backend/utils/adt/jsonfuncs.c | 164 ++++++++++++++++++--------
src/backend/utils/adt/jsonpath_exec.c | 36 ++++--
src/include/catalog/catversion.h | 3 +-
src/include/catalog/pg_proc.dat | 46 ++++++--
src/test/regress/expected/jsonb.out | 112 +++++++++++++++++-
src/test/regress/sql/jsonb.sql | 66 ++++++++++-
7 files changed, 501 insertions(+), 71 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..31656d8a11e 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2042,147 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = palloc0(sizeof(FuncExpr));
+ FuncExpr *jsonb_start_func = NULL, *jsonb_finish_func = NULL, *final_func = NULL;
+ Node *input;
+ Oid new_func_id = InvalidOid;
+ List *args;
+ Oid input_func_id, collid, inputcollid;
+ bool retset = false, variadic = false;
+
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
+ Assert(list_length(fexpr->args) == 1);
+ input = (Node *) linitial(fexpr->args);
+
+ if (IsA(input, OpExpr))
+ {
+ OpExpr *opExpr = castNode(OpExpr, input);
+ input_func_id = opExpr->opfuncid;
+ collid = opExpr->opcollid;
+ inputcollid = opExpr->inputcollid;
+ args = opExpr->args;
+ }
+ else if (IsA(input, FuncExpr))
+ {
+ FuncExpr *funcExpr = castNode(FuncExpr, input);
+ input_func_id = funcExpr->funcid;
+ collid = funcExpr->funccollid;
+ inputcollid = funcExpr->inputcollid;
+ args = funcExpr->args;
+ }
+ else
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
+
+
+ switch (input_func_id)
+ {
+ case F_JSONB_OBJECT_FIELD:
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ break;
+ case F_JSONB_ARRAY_ELEMENT:
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ break;
+ case F_JSONB_EXTRACT_PATH:
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+ variadic = true;
+ break;
+ case F_JSONB_PATH_QUERY:
+ new_func_id = F_JSONB_PATH_QUERY_START;
+ retset = true;
+ break;
+ case F_JSONB_PATH_QUERY_FIRST:
+ new_func_id = F_JSONB_PATH_QUERY_FIRST_START;
+ break;
+ default:
+ new_func_id = InvalidOid;
+ break;
+ }
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(fexpr);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, args,
+ collid, inputcollid,
+ COERCE_EXPLICIT_CALL);
+ jsonb_start_func->funcretset = retset;
+ jsonb_start_func->funcvariadic = variadic;
+
+ /* relabel the first arguments as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, 0,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make2(jsonb_start_func,
+ makeConst(OIDOID,
+ -1,
+ InvalidOid,
+ sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false,
+ true)),
+ collid, inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ final_func = (FuncExpr *)coerce_type(NULL, (Node *)jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), collid,
+ inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(fexpr);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ Oid final_oid = PG_GETARG_OID(1);
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, format_type_be(final_oid));
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..e3f6fea4e19 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -492,6 +492,7 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+static JsonbValue *jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull);
/*
* pg_parse_json_or_errsave
@@ -848,13 +849,12 @@ json_object_field(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_object_field(PG_FUNCTION_ARGS)
+static Datum
+jsonb_object_field_internal(FunctionCallInfo fcinfo, bool as_jsonb)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
text *key = PG_GETARG_TEXT_PP(1);
- JsonbValue *v;
- JsonbValue vbuf;
+ JsonbValue *v;
if (!JB_ROOT_IS_OBJECT(jb))
PG_RETURN_NULL();
@@ -862,12 +862,26 @@ jsonb_object_field(PG_FUNCTION_ARGS)
v = getKeyJsonValueFromContainer(&jb->root,
VARDATA_ANY(key),
VARSIZE_ANY_EXHDR(key),
- &vbuf);
+ NULL);
+ if (v == NULL)
+ PG_RETURN_NULL();
- if (v != NULL)
+ if (as_jsonb)
PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
- PG_RETURN_NULL();
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_object_field(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, true);
+}
+
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, false);
}
Datum
@@ -923,8 +937,8 @@ json_array_element(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_array_element(PG_FUNCTION_ARGS)
+static Datum
+jsonb_array_element_internal(FunctionCallInfo fcinfo, bool as_jsonb)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
int element = PG_GETARG_INT32(1);
@@ -945,10 +959,25 @@ jsonb_array_element(PG_FUNCTION_ARGS)
}
v = getIthJsonbValueFromContainer(&jb->root, element);
- if (v != NULL)
+ if (v == NULL)
+ PG_RETURN_NULL();
+
+ if (as_jsonb)
PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
- PG_RETURN_NULL();
+ PG_RETURN_POINTER(v);
+}
+
+Datum
+jsonb_array_element(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, true);
+}
+
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, false);
}
Datum
@@ -1473,6 +1502,39 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
return JSON_SUCCESS;
}
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
+
+ JsonbValue *v;
+
+ Datum *pathtext;
+ bool *pathnulls;
+ bool isnull = false;
+ int npath;
+
+ /*
+ * If the array contains any null elements, return NULL, on the grounds
+ * that you'd have gotten NULL if any RHS value were NULL in a nested
+ * series of applications of the -> operator. (Note: because we also
+ * return NULL for error cases such as no-such-field, this is true
+ * regardless of the contents of the rest of the array.)
+ */
+ if (array_contains_nulls(path))
+ PG_RETURN_NULL();
+
+ deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
+
+ v = jsonb_get_jsonbvalue(jb, pathtext, npath, &isnull);
+
+ if (isnull)
+ PG_RETURN_NULL();
+
+ PG_RETURN_POINTER(v);
+}
+
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
@@ -1516,52 +1578,36 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
PG_RETURN_DATUM(res);
}
-Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+
+static JsonbValue *
+jsonb_get_jsonbvalue(Jsonb *jb, Datum *path, int npath, bool *isnull)
{
+ bool have_object = false, have_array = false;
JsonbContainer *container = &jb->root;
+ int i;
JsonbValue *jbvp = NULL;
- int i;
- bool have_object = false,
- have_array = false;
- *isnull = false;
+ /*
+ * If the array is empty, return the entire LHS object, on the grounds
+ * that we should do zero field or element extractions.
+ */
+ if (npath <= 0)
+ {
+ JsonbValue *res = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ return getIthJsonbValueFromContainer(container, 0);
+
+ /* NB: res is a jbvBinary JsonbValue */
+ res = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, res);
+ return res;
+ }
/* Identify whether we have object, array, or scalar at top-level */
if (JB_ROOT_IS_OBJECT(jb))
have_object = true;
else if (JB_ROOT_IS_ARRAY(jb) && !JB_ROOT_IS_SCALAR(jb))
have_array = true;
- else
- {
- Assert(JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb));
- /* Extract the scalar value, if it is what we'll return */
- if (npath <= 0)
- jbvp = getIthJsonbValueFromContainer(container, 0);
- }
-
- /*
- * If the array is empty, return the entire LHS object, on the grounds
- * that we should do zero field or element extractions. For the
- * non-scalar case we can just hand back the object without much work. For
- * the scalar case, fall through and deal with the value below the loop.
- * (This inconsistency arises because there's no easy way to generate a
- * JsonbValue directly for root-level containers.)
- */
- if (npath <= 0 && jbvp == NULL)
- {
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
- }
- }
for (i = 0; i < npath; i++)
{
@@ -1586,7 +1632,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (endptr == indextext || *endptr != '\0' || errno != 0)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (lindex >= 0)
@@ -1607,7 +1653,7 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
if (lindex == INT_MIN || -lindex > nelements)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else
index = nelements + lindex;
@@ -1619,13 +1665,13 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
{
/* scalar, extraction yields a null */
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
if (jbvp == NULL)
{
*isnull = true;
- return PointerGetDatum(NULL);
+ return NULL;
}
else if (i == npath - 1)
break;
@@ -1644,6 +1690,22 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
have_array = false;
}
}
+ return jbvp;
+}
+
+/*
+ * Return jsonb datum or jsonb-as-text datum.
+ */
+Datum
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+{
+ JsonbValue *jbvp = NULL;
+ *isnull = false;
+
+ jbvp = jsonb_get_jsonbvalue(jb, path, npath, isnull);
+
+ if (*isnull)
+ return PointerGetDatum(NULL);
if (as_text)
{
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 2d0599b4aaa..973adb9981b 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -393,7 +393,7 @@ jsonb_path_match_opr(PG_FUNCTION_ARGS)
* rowset.
*/
static Datum
-jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz, bool as_jsonb)
{
FuncCallContext *funcctx;
List *found;
@@ -435,19 +435,28 @@ jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
v = lfirst(c);
funcctx->user_fctx = list_delete_first(found);
- SRF_RETURN_NEXT(funcctx, JsonbPGetDatum(JsonbValueToJsonb(v)));
+ if (as_jsonb)
+ SRF_RETURN_NEXT(funcctx, JsonbPGetDatum(JsonbValueToJsonb(v)));
+ else
+ SRF_RETURN_NEXT(funcctx, PointerGetDatum(v));
}
Datum
jsonb_path_query(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, false);
+ return jsonb_path_query_internal(fcinfo, false, true);
}
Datum
jsonb_path_query_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, true);
+ return jsonb_path_query_internal(fcinfo, true, true);
+}
+
+Datum
+jsonb_path_query_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_internal(fcinfo, false, false);
}
/*
@@ -487,7 +496,7 @@ jsonb_path_query_array_tz(PG_FUNCTION_ARGS)
* item. If there are no items, NULL returned.
*/
static Datum
-jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz, bool as_jsonb)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
JsonPath *jp = PG_GETARG_JSONPATH_P(1);
@@ -498,7 +507,12 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
(void) executeJsonPath(jp, vars, jb, !silent, &found, tz);
if (JsonValueListLength(&found) >= 1)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(JsonValueListHead(&found)));
+ {
+ if (as_jsonb)
+ PG_RETURN_JSONB_P(JsonbValueToJsonb(JsonValueListHead(&found)));
+ else
+ PG_RETURN_POINTER(JsonValueListHead(&found));
+ }
else
PG_RETURN_NULL();
}
@@ -506,13 +520,19 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
Datum
jsonb_path_query_first(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, false);
+ return jsonb_path_query_first_internal(fcinfo, false, true);
}
Datum
jsonb_path_query_first_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, true);
+ return jsonb_path_query_first_internal(fcinfo, true, true);
+}
+
+Datum
+jsonb_path_query_first_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_first_internal(fcinfo, false, false);
}
/********************Execute functions for JsonPath**************************/
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index ab9a7ac1f79..1bf0b828853 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,5 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308251
-
+#define CATALOG_VERSION_NO 202309041
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9805bc61180..ef719aac6bb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4587,25 +4587,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9947,6 +9947,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552', descr => 'extract jsonbvalue from jsonb for the given field',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '3813', descr => 'extract josnbvalue from jsonb array for the given index',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4549', descr => 'extract jsonbvalue from jsonb for the given paths',
+ proname => 'jsonb_extract_path_start', provariadic => 'text', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{internal,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553', descr => 'convert a jsonbvalue to numeric',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal oid', proargnames => '{from_jsonvalue,target_oid}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554', descr => 'convert a jsonbvalue to boolean',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -10139,6 +10163,10 @@
proname => 'jsonb_path_query', prorows => '1000', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query' },
+{ oid => '4557', descr => 'jsonpath query as jsonbvalue',
+ proname => 'jsonb_path_query_start', prorows => '1000', proretset => 't',
+ prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_start' },
{ oid => '4007', descr => 'jsonpath query wrapped into array',
proname => 'jsonb_path_query_array', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
@@ -10147,6 +10175,10 @@
proname => 'jsonb_path_query_first', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query_first' },
+{ oid => '4555', descr => 'jsonpath query first item as jsonbvalue',
+ proname => 'jsonb_path_query_first_start', prorettype => 'internal',
+ proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_first_start' },
{ oid => '4009', descr => 'jsonpath match',
proname => 'jsonb_path_match', prorettype => 'bool',
proargtypes => 'jsonb jsonpath jsonb bool', prosrc => 'jsonb_path_match' },
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..b612fb85ec2 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,114 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '1700'::oid), (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '23'::oid))::integer, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '20'::oid))::bigint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '700'::oid))::real, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field4'::text), '701'::oid))::double precision, jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 0), '1700'::oid), jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal(0), 10), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), VARIADIC '{field6,f1}'::text[]), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), VARIADIC '{field6,f2}'::text[]), '1700'::oid), jsonb_finish_bool(jsonb_extract_path_start((test_json)::internal(0), VARIADIC '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Result
+ Output: (jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), jsonb_path_query_first(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), (jsonb_finish_numeric((jsonb_path_query_start((test_json)::internal(0), '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_path_query_first_start((test_json)::internal(0), '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), '21'::oid))::smallint
+ -> ProjectSet
+ Output: jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), jsonb_path_query_start((test_json)::internal(0), '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), test_json
+ -> Seq Scan on pg_temp.test_jsonb
+ Output: json_type, test_json
+ Filter: (test_jsonb.json_type = 'object'::text)
+(7 rows)
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ jsonb_path_query | jsonb_path_query_first | jsonb_path_query | jsonb_path_query_first
+------------------+------------------------+------------------+------------------------
+ 2 | 2 | 2 | 2
+ 3 | 2 | 3 | 2
+ 4 | 2 | 4 | 2
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal(0), VARIADIC '{}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal(0), 'field1'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type integer
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +692,9 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+ field8
+(8 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..500d04936d9 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,72 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-05 12:51 ` jian he <[email protected]>
2023-09-06 07:00 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: jian he @ 2023-09-05 12:51 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: Chapman Flack <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Mon, Sep 4, 2023 at 10:35 PM Andy Fan <[email protected]> wrote:
>
> Hi,
>
> v13 attached. Changes includes:
>
> 1. fix the bug Jian provides.
> 2. reduce more code duplication without DirectFunctionCall.
> 3. add the overlooked jsonb_path_query and jsonb_path_query_first as candidates
>
>
> --
> Best Regards
> Andy Fan
based on v13.
IMHO, it might be a good idea to write some comments on
jsonb_object_field_internal. especially the second boolean argument.
something like "some case, we just want return JsonbValue rather than
Jsonb. to return JsonbValue, make as_jsonb be false".
I am not sure "jsonb_object_field_start" is a good name, so far I only
come up with "jsonb_object_field_to_jsonbvalues".
linitial(jsonb_start_func->args) =
makeRelabelType(linitial(jsonb_start_func->args),
INTERNALOID, 0,
InvalidOid,
COERCE_IMPLICIT_CAST);
if no need, output typmod (usually -1), so here should be -1 rather than 0?
list_make2(jsonb_start_func, makeConst(.....). you can just combine
two different types then make a list, seems pretty cool...
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-05 12:51 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
@ 2023-09-06 07:00 ` Andy Fan <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Andy Fan @ 2023-09-06 07:00 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Chapman Flack <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
>
>
> based on v13.
> IMHO, it might be a good idea to write some comments on
> jsonb_object_field_internal. especially the second boolean argument.
> something like "some case, we just want return JsonbValue rather than
> Jsonb. to return JsonbValue, make as_jsonb be false".
>
OK, I will proposal "return a JsonbValue when as_jsonb is false".
> I am not sure "jsonb_object_field_start" is a good name, so far I only
> come up with "jsonb_object_field_to_jsonbvalues".
Yes, I think it is a good idea. Puting the jsonbvalue in the name can
compensate for the imprecision of "internal" as a return type. I am
thinking
if we should rename jsonb_finish_numeric to jsonbvalue_to_numeric as
well.
>
linitial(jsonb_start_func->args) =
> makeRelabelType(linitial(jsonb_start_func->args),
> INTERNALOID, 0,
> InvalidOid,
> COERCE_IMPLICIT_CAST);
>
> if no need, output typmod (usually -1), so here should be -1 rather than 0?
I agree. -1 is better than 0.
Thanks for the code level review again! I want to wait for some longer time
to gather more feedback. I'm willing to name it better, but hope I didn't
rename it to A and rename it back shortly.
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-13 21:18 ` Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-09-13 21:18 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2023-09-04 10:35, Andy Fan wrote:
> v13 attached. Changes includes:
>
> 1. fix the bug Jian provides.
> 2. reduce more code duplication without DirectFunctionCall.
> 3. add the overlooked jsonb_path_query and jsonb_path_query_first as
> candidates
Apologies for the delay. I like the way this is shaping up.
My first comment will be one that may be too large for this patch
(and too large to rest on my opinion alone); that's why I'm making
it first.
It seems at first a minor point: to me it feels like a wart to have
to pass jsonb_finish_numeric (and *only* jsonb_finish_numeric) a type
oid reflecting the target type of a cast that's going to be applied
*after jsonb_finish_numeric has done its work*, and only for the
purpose of generating a message if the jsonb type *isn't numeric*,
but saying "cannot cast to" (that later target type) instead.
I understand this is done to exactly match the existing behavior,
so what makes this a larger issue is I'm not convinced the existing
behavior is good. Therefore I'm not convinced that bending over
backward to preserve it is good.
What's not good: the places a message from cannotCastJsonbValue
are produced, there has been no attempt yet to cast anything.
The message purely tells you about whether you have the kind
of jsonb node you think you have (and array, bool, null, numeric,
object, string are the only kinds of those). If you're wrong
about what kind of jsonb node it is, you get this message.
If you're right about the kind of node, you don't get this
message, regardless of whether its value can be cast to the
later target type. For example, '32768'::jsonb::int2 produces
ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE "smallint out of range"
but that message comes from the actual int2 cast.
IMV, what the "cannot cast jsonb foo to type %s" message really
means is "jsonb foo where jsonb bar is required" and that's what
it should say, and that message depends on nothing about any
future plans for what will be done to the jsonb bar, so it can
be produced without needing any extra information to be passed.
I'm also not convinced ERRCODE_INVALID_PARAMETER_VALUE is a
good errcode for that message (whatever the wording). I do not
see much precedent elsewhere in the code for using
INVALID_PARAMETER_VALUE to signal this kind of "data value
isn't what you think it is" condition. Mostly it is used
when checking the kinds of parameters passed to a function to
indicate what it should do.
There seem to be several more likely choices for an errcode
there in the 2203x range.
But I understand that issue is not with this patch so much
as with preexisting behavior, and because it's preexisting,
there can be sound arguments against changing it.
But if that preexisting message could be changed, it would
eliminate the need for an unpleasing wart here.
Other notes are more minor:
+ else
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(fexpr);
...
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(fexpr);
...
+ default:
+ PG_RETURN_POINTER(fexpr);
If I am reading supportnodes.h right, returning NULL is
sufficient to say no transformation is needed.
+ FuncExpr *fexpr = palloc0(sizeof(FuncExpr));
...
+ memcpy(fexpr, req->fcall, sizeof(FuncExpr));
Is the shallow copy necessary? If so, a comment explaining why
might be apropos. Because the copy is shallow, if there is any
concern about the lifespan of req->fcall, would there not be a
concern about its children?
Is there a reason not to transform the _tz flavors of
jsonb_path_query and jsonb_path-query_first?
- JsonbValue *v;
- JsonbValue vbuf;
+ JsonbValue *v;
...
+ int i;
JsonbValue *jbvp = NULL;
- int i;
Sometimes it's worth looking over a patch for changes like these,
and reverting such whitespace or position changes, if they aren't
things you want a reviewer to be squinting at. :)
Regards,
-Chap
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-09-14 06:41 ` Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-09-14 06:41 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, Sep 14, 2023 at 5:18 AM Chapman Flack <[email protected]> wrote:
> On 2023-09-04 10:35, Andy Fan wrote:
> > v13 attached. Changes includes:
> >
> > 1. fix the bug Jian provides.
> > 2. reduce more code duplication without DirectFunctionCall.
> > 3. add the overlooked jsonb_path_query and jsonb_path_query_first as
> > candidates
>
> Apologies for the delay. I like the way this is shaping up.
This is a great signal:)
>
>
My first comment will be one that may be too large for this patch
> (and too large to rest on my opinion alone); that's why I'm making
> it first.
>
> It seems at first a minor point: to me it feels like a wart to have
> to pass jsonb_finish_numeric (and *only* jsonb_finish_numeric) a type
> oid reflecting the target type of a cast that's going to be applied
> *after jsonb_finish_numeric has done its work*, and only for the
> purpose of generating a message if the jsonb type *isn't numeric*,
> but saying "cannot cast to" (that later target type) instead.
>
> I understand this is done to exactly match the existing behavior,
> so what makes this a larger issue is I'm not convinced the existing
> behavior is good. Therefore I'm not convinced that bending over
> backward to preserve it is good.
>
I hesitated to do so, but I'm thinking if any postgresql user uses
some code like if (errMsg.equals('old-error-message')), and if we
change the error message, the application will be broken. I agree
with the place for the error message, IIUC, you intend to choose
not compatible with the old error message?
What's not good: the places a message from cannotCastJsonbValue
> are produced, there has been no attempt yet to cast anything.
> The message purely tells you about whether you have the kind
> of jsonb node you think you have (and array, bool, null, numeric,
> object, string are the only kinds of those). If you're wrong
> about what kind of jsonb node it is, you get this message.
> If you're right about the kind of node, you don't get this
> message, regardless of whether its value can be cast to the
> later target type. For example, '32768'::jsonb::int2 produces
> ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE "smallint out of range"
> but that message comes from the actual int2 cast.
>
> IMV, what the "cannot cast jsonb foo to type %s" message really
> means is "jsonb foo where jsonb bar is required" and that's what
> it should say, and that message depends on nothing about any
> future plans for what will be done to the jsonb bar, so it can
> be produced without needing any extra information to be passed.
>
> I'm also not convinced ERRCODE_INVALID_PARAMETER_VALUE is a
> good errcode for that message (whatever the wording). I do not
> see much precedent elsewhere in the code for using
> INVALID_PARAMETER_VALUE to signal this kind of "data value
> isn't what you think it is" condition. Mostly it is used
> when checking the kinds of parameters passed to a function to
> indicate what it should do.
>
> There seem to be several more likely choices for an errcode
> there in the 2203x range.
>
> But I understand that issue is not with this patch so much
> as with preexisting behavior, and because it's preexisting,
> there can be sound arguments against changing it.
> But if that preexisting message could be changed, it would
> eliminate the need for an unpleasing wart here.
>
> Other notes are more minor:
>
> + else
> + /* not the desired pattern. */
> + PG_RETURN_POINTER(fexpr);
> ...
> +
> + if (!OidIsValid(new_func_id))
> + PG_RETURN_POINTER(fexpr);
> ...
> + default:
> + PG_RETURN_POINTER(fexpr);
>
> If I am reading supportnodes.h right, returning NULL is
> sufficient to say no transformation is needed.
>
I double confirmed you are right here.
Changed it to PG_RETURN_POINT(null); here in the next version.
>
> + FuncExpr *fexpr = palloc0(sizeof(FuncExpr));
> ...
> + memcpy(fexpr, req->fcall, sizeof(FuncExpr));
>
> Is the shallow copy necessary? If so, a comment explaining why
> might be apropos. Because the copy is shallow, if there is any
> concern about the lifespan of req->fcall, would there not be a
> concern about its children?
>
All the interesting parts in the input FuncExpr are heap based,
but the FuncExpr itself is stack based (I'm not sure why the fact
works like this), Also based on my previously understanding, I
need to return a FuncExpr original even if the function can't be
simplified, so I made a shallow copy. There will be no copy at
all in the following version since I returned NULL in such a case.
> Is there a reason not to transform the _tz flavors of
> jsonb_path_query and jsonb_path-query_first?
>
I misunderstood the _tz flavors return timestamp, after some deep
reading of these functions, they just work at the comparisons part.
so I will add them in the following version.
>
> - JsonbValue *v;
> - JsonbValue vbuf;
> + JsonbValue *v;
> ...
> + int i;
> JsonbValue *jbvp = NULL;
> - int i;
>
> Sometimes it's worth looking over a patch for changes like these,
> and reverting such whitespace or position changes, if they aren't
> things you want a reviewer to be squinting at. :)
>
Yes, I look over my patch carefully before sending it out usually,
but this is an oversight.
Lastly, do you have any idea about the function naming as Jian & I
discussed at [1]?
[1]
https://www.postgresql.org/message-id/CAKU4AWqQ0hed%3DZmpT%2B7Vxnp4H9ZxQqFz30%3Dk%3DvvrmNe57X4dKQ%40...
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-09-15 01:53 ` Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-09-15 01:53 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> Is there a reason not to transform the _tz flavors of
>> jsonb_path_query and jsonb_path-query_first?
>>
>
> I misunderstood the _tz flavors return timestamp, after some deep
> reading of these functions, they just work at the comparisons part.
> so I will add them in the following version.
>
_tz favors did return timestamp.. the reason is stated in the commit
messge of patch 2.
try to apply jsonb extraction optimization to _tz functions.
both jsonb_path_query_tz and jsonb_path_query_tz_first returns
the elements which are timestamp comparable, but such elements
are impossible to be cast to numeric or boolean IIUC. Just provides
this commit for communication purpose only.
so v14 is attached, changes include:
1. Change the typmod for internal type from 0 to -1.
2. return NULL for non-simplify expressions from the planner
support function, hence shallow copy is removed as well.
Things are not addressed yet:
1. the error message handling.
2. if we have chances to optimize _tz functions, I guess no.
3. function naming issue. I think I can get it modified once after
all the other issues are addressed.
--
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v14-0002-try-to-apply-jsonb-extraction-optimization-to-_t.patch (3.9K, 3-v14-0002-try-to-apply-jsonb-extraction-optimization-to-_t.patch)
download | inline diff:
From 29a2ca508541628df9e902be60b3ec1ea81424cf Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Fri, 15 Sep 2023 09:37:32 +0800
Subject: [PATCH v14 2/2] try to apply jsonb extraction optimization to _tz
functions.
both jsonb_path_query_tz and jsonb_path_query_tz_first returns
the element which is timestamp comparable, but such element is
impossible to be cast to numeric or boolean IIUC. Just provides
this commit for communication purpose only.
---
src/backend/utils/adt/jsonb.c | 8 +++++++-
src/backend/utils/adt/jsonpath_exec.c | 12 ++++++++++++
src/include/catalog/pg_proc.dat | 8 ++++++++
3 files changed, 27 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index e2efb28a685..aad3b7f542f 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -2089,7 +2089,6 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
/* not the desired pattern. */
PG_RETURN_POINTER(NULL);
-
switch (input_func_id)
{
case F_JSONB_OBJECT_FIELD:
@@ -2106,9 +2105,16 @@ jsonb_cast_support(PG_FUNCTION_ARGS)
new_func_id = F_JSONB_PATH_QUERY_START;
retset = true;
break;
+ case F_JSONB_PATH_QUERY_TZ:
+ new_func_id = F_JSONB_PATH_QUERY_TZ_START;
+ retset = true;
+ break;
case F_JSONB_PATH_QUERY_FIRST:
new_func_id = F_JSONB_PATH_QUERY_FIRST_START;
break;
+ case F_JSONB_PATH_QUERY_FIRST_TZ:
+ new_func_id = F_JSONB_PATH_QUERY_FIRST_TZ_START;
+ break;
default:
new_func_id = InvalidOid;
break;
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 5eef14f93ed..9b7108418be 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -456,6 +456,12 @@ jsonb_path_query_start(PG_FUNCTION_ARGS)
return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
+Datum
+jsonb_path_query_tz_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_internal(fcinfo, true, JsonbValue_AsJsonbValue);
+}
+
/*
* jsonb_path_query_array
* Executes jsonpath for given jsonb document and returns result as
@@ -530,6 +536,12 @@ jsonb_path_query_first_start(PG_FUNCTION_ARGS)
return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
+Datum
+jsonb_path_query_first_tz_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_first_internal(fcinfo, true, JsonbValue_AsJsonbValue);
+}
+
/********************Execute functions for JsonPath**************************/
/*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ef719aac6bb..7e76d611c06 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10167,6 +10167,10 @@
proname => 'jsonb_path_query_start', prorows => '1000', proretset => 't',
prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
prosrc => 'jsonb_path_query_start' },
+{ oid => '4556', descr => 'jsonpath query tz as jsonbvalue',
+ proname => 'jsonb_path_query_tz_start', prorows => '1000', proretset => 't',
+ prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_tz_start' },
{ oid => '4007', descr => 'jsonpath query wrapped into array',
proname => 'jsonb_path_query_array', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
@@ -10179,6 +10183,10 @@
proname => 'jsonb_path_query_first_start', prorettype => 'internal',
proargtypes => 'internal jsonpath jsonb bool',
prosrc => 'jsonb_path_query_first_start' },
+{ oid => '4551', descr => 'jsonpath query tz first item as jsonbvalue',
+ proname => 'jsonb_path_query_first_tz_start', prorettype => 'internal',
+ proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_first_tz_start'},
{ oid => '4009', descr => 'jsonpath match',
proname => 'jsonb_path_match', prorettype => 'bool',
proargtypes => 'jsonb jsonpath jsonb bool', prosrc => 'jsonb_path_match' },
--
2.21.0
[application/octet-stream] v14-0001-optimize-casting-jsonb-to-a-given-type.patch (36.3K, 4-v14-0001-optimize-casting-jsonb-to-a-given-type.patch)
download | inline diff:
From ff58d54ca52d87a2594793c7a904fa9982c95374 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Thu, 14 Sep 2023 16:58:13 +0800
Subject: [PATCH v14 1/2] optimize casting jsonb to a given type.
Previously after we get a JsonbValue, we need to convert it to
Jsonb first then cast the Jsonb to the given type. In this patch,
we covert the JsonbValue to the desired type directly.
---
src/backend/utils/adt/jsonb.c | 152 ++++++++++++++++++++++++++
src/backend/utils/adt/jsonbsubs.c | 4 +-
src/backend/utils/adt/jsonfuncs.c | 125 ++++++++++++++-------
src/backend/utils/adt/jsonpath_exec.c | 31 ++++--
src/include/catalog/catversion.h | 3 +-
src/include/catalog/pg_proc.dat | 46 ++++++--
src/include/utils/jsonb.h | 11 +-
src/test/regress/expected/jsonb.out | 112 ++++++++++++++++++-
src/test/regress/sql/jsonb.sql | 66 ++++++++++-
9 files changed, 485 insertions(+), 65 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9781852b0cb..e2efb28a685 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2038,6 +2042,154 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+
+/*
+ * jsonb_cast_support()
+ *
+ * Planner support function for casting a jsonb extraction to a numeric
+ * or bool data type. Instead of converting a jsonbvalue to jsonb, the new
+ * method will cast the jsonbvalue to the desired data type directly.
+ */
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = req->fcall;
+ FuncExpr *jsonb_start_func = NULL, *jsonb_finish_func = NULL, *final_func = NULL;
+ Node *input;
+ Oid new_func_id = InvalidOid;
+ List *args;
+ Oid input_func_id, collid, inputcollid;
+ bool retset = false, variadic = false;
+
+ Assert(list_length(fexpr->args) == 1);
+ input = (Node *) linitial(fexpr->args);
+
+ if (IsA(input, OpExpr))
+ {
+ OpExpr *opExpr = castNode(OpExpr, input);
+ input_func_id = opExpr->opfuncid;
+ collid = opExpr->opcollid;
+ inputcollid = opExpr->inputcollid;
+ args = opExpr->args;
+ }
+ else if (IsA(input, FuncExpr))
+ {
+ FuncExpr *funcExpr = castNode(FuncExpr, input);
+ input_func_id = funcExpr->funcid;
+ collid = funcExpr->funccollid;
+ inputcollid = funcExpr->inputcollid;
+ args = funcExpr->args;
+ }
+ else
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(NULL);
+
+
+ switch (input_func_id)
+ {
+ case F_JSONB_OBJECT_FIELD:
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ break;
+ case F_JSONB_ARRAY_ELEMENT:
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ break;
+ case F_JSONB_EXTRACT_PATH:
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+ variadic = true;
+ break;
+ case F_JSONB_PATH_QUERY:
+ new_func_id = F_JSONB_PATH_QUERY_START;
+ retset = true;
+ break;
+ case F_JSONB_PATH_QUERY_FIRST:
+ new_func_id = F_JSONB_PATH_QUERY_FIRST_START;
+ break;
+ default:
+ new_func_id = InvalidOid;
+ break;
+ }
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(NULL);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, args,
+ collid, inputcollid,
+ COERCE_EXPLICIT_CALL);
+ jsonb_start_func->funcretset = retset;
+ jsonb_start_func->funcvariadic = variadic;
+
+ /* relabel the first arguments as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, -1,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make2(jsonb_start_func,
+ makeConst(OIDOID,
+ -1,
+ InvalidOid,
+ sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false,
+ true)),
+ collid, inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ final_func = (FuncExpr *)coerce_type(NULL, (Node *)jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), collid,
+ inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(NULL);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ Oid final_oid = PG_GETARG_OID(1);
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, format_type_be(final_oid));
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *)PG_GETARG_POINTER(0);
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonbsubs.c b/src/backend/utils/adt/jsonbsubs.c
index de0ae3604ff..cb050c65ef7 100644
--- a/src/backend/utils/adt/jsonbsubs.c
+++ b/src/backend/utils/adt/jsonbsubs.c
@@ -251,7 +251,7 @@ jsonb_subscript_fetch(ExprState *state,
workspace->index,
sbsrefstate->numupper,
op->resnull,
- false);
+ JsonbValue_AsJsonb);
}
/*
@@ -343,7 +343,7 @@ jsonb_subscript_fetch_old(ExprState *state,
sbsrefstate->upperindex,
sbsrefstate->numupper,
&sbsrefstate->prevnull,
- false);
+ JsonbValue_AsJsonb);
}
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index a4bfa5e4040..eded8e4f32b 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -357,7 +357,7 @@ static JsonParseErrorType get_scalar(void *state, char *token, JsonTokenType tok
static Datum get_path_all(FunctionCallInfo fcinfo, bool as_text);
static text *get_worker(text *json, char **tpath, int *ipath, int npath,
bool normalize_results);
-static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text);
+static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target);
static text *JsonbValueAsText(JsonbValue *v);
/* semantic action functions for json_array_length */
@@ -492,6 +492,20 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+Datum
+jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target)
+{
+ switch (target)
+ {
+ case JsonbValue_AsJsonbValue:
+ PG_RETURN_POINTER(jbv);
+ case JsonbValue_AsJsonb:
+ PG_RETURN_JSONB_P(JsonbValueToJsonb(jbv));
+ case JsonbValue_AsText:
+ PG_RETURN_TEXT_P(JsonbValueAsText(jbv));
+ }
+ PG_RETURN_POINTER(NULL);
+}
/*
* pg_parse_json_or_errsave
@@ -848,13 +862,12 @@ json_object_field(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_object_field(PG_FUNCTION_ARGS)
+static Datum
+jsonb_object_field_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
text *key = PG_GETARG_TEXT_PP(1);
JsonbValue *v;
- JsonbValue vbuf;
if (!JB_ROOT_IS_OBJECT(jb))
PG_RETURN_NULL();
@@ -862,12 +875,23 @@ jsonb_object_field(PG_FUNCTION_ARGS)
v = getKeyJsonValueFromContainer(&jb->root,
VARDATA_ANY(key),
VARSIZE_ANY_EXHDR(key),
- &vbuf);
+ NULL);
+ if (v == NULL)
+ PG_RETURN_NULL();
- if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
+ return jsonbvalue_covert(v, target);
+}
- PG_RETURN_NULL();
+Datum
+jsonb_object_field(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonbValue);
}
Datum
@@ -923,8 +947,8 @@ json_array_element(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_array_element(PG_FUNCTION_ARGS)
+static Datum
+jsonb_array_element_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
int element = PG_GETARG_INT32(1);
@@ -945,10 +969,22 @@ jsonb_array_element(PG_FUNCTION_ARGS)
}
v = getIthJsonbValueFromContainer(&jb->root, element);
- if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
+ if (v == NULL)
+ PG_RETURN_NULL();
- PG_RETURN_NULL();
+ return jsonbvalue_covert(v, target);
+}
+
+Datum
+jsonb_array_element(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonbValue);
}
Datum
@@ -1476,17 +1512,23 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, false);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonb);
}
Datum
jsonb_extract_path_text(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, true);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsText);
+}
+
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonbValue);
}
static Datum
-get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
+get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
@@ -1508,7 +1550,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
- res = jsonb_get_element(jb, pathtext, npath, &isnull, as_text);
+ res = jsonb_get_element(jb, pathtext, npath, &isnull, target);
if (isnull)
PG_RETURN_NULL();
@@ -1517,7 +1559,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
}
Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, JsonbValueTarget target)
{
JsonbContainer *container = &jb->root;
JsonbValue *jbvp = NULL;
@@ -1550,16 +1592,26 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
*/
if (npath <= 0 && jbvp == NULL)
{
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
+ switch(target)
{
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsText:
+ return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
+ container,
+ VARSIZE(jb))));
+ case JsonbValue_AsJsonb:
+ /* just hand back the jsonb */
+ PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsJsonbValue:
+ {
+ /* just hand back the jsonb */
+ JsonbValue *jbv = NULL;
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ PG_RETURN_POINTER(getIthJsonbValueFromContainer(container, 0));
+
+ jbv = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, jbv);
+ PG_RETURN_POINTER(jbv);
+ }
}
}
@@ -1645,23 +1697,14 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
}
}
- if (as_text)
- {
- if (jbvp->type == jbvNull)
- {
- *isnull = true;
- return PointerGetDatum(NULL);
- }
- return PointerGetDatum(JsonbValueAsText(jbvp));
- }
- else
+ if (target == JsonbValue_AsText && jbvp->type == jbvNull)
{
- Jsonb *res = JsonbValueToJsonb(jbvp);
-
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(res);
+ *isnull = true;
+ return PointerGetDatum(NULL);
}
+
+ return jsonbvalue_covert(jbvp, target);
}
Datum
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 2d0599b4aaa..5eef14f93ed 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -393,7 +393,7 @@ jsonb_path_match_opr(PG_FUNCTION_ARGS)
* rowset.
*/
static Datum
-jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
FuncCallContext *funcctx;
List *found;
@@ -435,19 +435,25 @@ jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
v = lfirst(c);
funcctx->user_fctx = list_delete_first(found);
- SRF_RETURN_NEXT(funcctx, JsonbPGetDatum(JsonbValueToJsonb(v)));
+ SRF_RETURN_NEXT(funcctx, jsonbvalue_covert(v, target));
}
Datum
jsonb_path_query(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, false);
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, true);
+ return jsonb_path_query_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/*
@@ -487,7 +493,7 @@ jsonb_path_query_array_tz(PG_FUNCTION_ARGS)
* item. If there are no items, NULL returned.
*/
static Datum
-jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
JsonPath *jp = PG_GETARG_JSONPATH_P(1);
@@ -498,7 +504,10 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
(void) executeJsonPath(jp, vars, jb, !silent, &found, tz);
if (JsonValueListLength(&found) >= 1)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(JsonValueListHead(&found)));
+ {
+ JsonbValue *jbv = JsonValueListHead(&found);
+ return jsonbvalue_covert(jbv, target);
+ }
else
PG_RETURN_NULL();
}
@@ -506,13 +515,19 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
Datum
jsonb_path_query_first(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, false);
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_first_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, true);
+ return jsonb_path_query_first_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_first_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/********************Execute functions for JsonPath**************************/
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index ab9a7ac1f79..fb83a7a29df 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,5 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202308251
-
+#define CATALOG_VERSION_NO 202309151
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9805bc61180..ef719aac6bb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4587,25 +4587,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9947,6 +9947,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552', descr => 'extract jsonbvalue from jsonb for the given field',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '3813', descr => 'extract josnbvalue from jsonb array for the given index',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4549', descr => 'extract jsonbvalue from jsonb for the given paths',
+ proname => 'jsonb_extract_path_start', provariadic => 'text', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{internal,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553', descr => 'convert a jsonbvalue to numeric',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal oid', proargnames => '{from_jsonvalue,target_oid}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554', descr => 'convert a jsonbvalue to boolean',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -10139,6 +10163,10 @@
proname => 'jsonb_path_query', prorows => '1000', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query' },
+{ oid => '4557', descr => 'jsonpath query as jsonbvalue',
+ proname => 'jsonb_path_query_start', prorows => '1000', proretset => 't',
+ prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_start' },
{ oid => '4007', descr => 'jsonpath query wrapped into array',
proname => 'jsonb_path_query_array', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
@@ -10147,6 +10175,10 @@
proname => 'jsonb_path_query_first', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query_first' },
+{ oid => '4555', descr => 'jsonpath query first item as jsonbvalue',
+ proname => 'jsonb_path_query_first_start', prorettype => 'internal',
+ proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_first_start' },
{ oid => '4009', descr => 'jsonpath match',
proname => 'jsonb_path_match', prorettype => 'bool',
proargtypes => 'jsonb jsonpath jsonb bool', prosrc => 'jsonb_path_match' },
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index 649a1644f24..50b3c65557f 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -294,6 +294,13 @@ struct JsonbValue
} val;
};
+typedef enum JsonbValueTarget
+{
+ JsonbValue_AsJsonbValue,
+ JsonbValue_AsJsonb,
+ JsonbValue_AsText
+} JsonbValueTarget;
+
#define IsAJsonbScalar(jsonbval) (((jsonbval)->type >= jbvNull && \
(jsonbval)->type <= jbvBool) || \
(jsonbval)->type == jbvDatetime)
@@ -428,12 +435,12 @@ extern const char *JsonbTypeName(JsonbValue *val);
extern Datum jsonb_set_element(Jsonb *jb, Datum *path, int path_len,
JsonbValue *newval);
extern Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath,
- bool *isnull, bool as_text);
+ bool *isnull, JsonbValueTarget target);
extern bool to_jsonb_is_immutable(Oid typoid);
extern Datum jsonb_build_object_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, Datum *args, bool *nulls,
Oid *types, bool absent_on_null);
-
+extern Datum jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbafb..c6af7a59ccb 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,114 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '1700'::oid), (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '23'::oid))::integer, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '20'::oid))::bigint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '700'::oid))::real, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '701'::oid))::double precision, jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 0), '1700'::oid), jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 10), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f1}'::text[]), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f2}'::text[]), '1700'::oid), jsonb_finish_bool(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Result
+ Output: (jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), jsonb_path_query_first(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), (jsonb_finish_numeric((jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_path_query_first_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), '21'::oid))::smallint
+ -> ProjectSet
+ Output: jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), test_json
+ -> Seq Scan on pg_temp.test_jsonb
+ Output: json_type, test_json
+ Filter: (test_jsonb.json_type = 'object'::text)
+(7 rows)
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ jsonb_path_query | jsonb_path_query_first | jsonb_path_query | jsonb_path_query_first
+------------------+------------------------+------------------+------------------------
+ 2 | 2 | 2 | 2
+ 3 | 2 | 3 | 2
+ 4 | 2 | 4 | 2
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field1'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type integer
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +692,9 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+ field8
+(8 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703d..500d04936d9 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,72 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
--
2.21.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-10-05 06:53 ` Andy Fan <[email protected]>
2023-11-01 02:17 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2023-10-05 06:53 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: jian he <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hi,
I am feeling this topic has been well discussed and the only pending
issues are below, it would be great that any committer can have a
look at these, so I mark this entry as "Ready for Committer".
Things are not addressed yet:
> 1. the error message handling.
>
You can check [1] for more background of this, I think blocking this
feature at an error message level is not pretty reasonable.
> 2. if we have chances to optimize _tz functions, I guess no.
>
patch 002 is dedicated for this, I think it should not be committed,
the reason is described in the commit message.
3. function naming issue. I think I can get it modified once after
> all the other issues are addressed.
>
>
[1]
https://www.postgresql.org/message-id/d70280648894e56f9f0d12c75090c3d8%40anastigmatix.net
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2023-11-01 02:17 ` Chapman Flack <[email protected]>
2023-11-06 03:26 ` Re: Extract numeric filed in JSONB more effectively [email protected]
0 siblings, 1 reply; 35+ messages in thread
From: Chapman Flack @ 2023-11-01 02:17 UTC (permalink / raw)
To: [email protected]; +Cc: Andy Fan <[email protected]>
Adding this comment via the CF app so it isn't lost, while an improperly-interpreted-DKIM-headers issue is still preventing me from mailing directly to -hackers.
It was my view that the patch was getting close by the end of the last commitfest, but still contained a bit of a logic wart made necessary by a questionable choice of error message wording, such that in my view it would be better to determine whether a different error message would better conform to ISO SQL in the first place, and obviate the need for the logic wart.
There seemed to be some progress possible on that when petere had time to weigh in on the standard shortly after the last CF ended.
So, it would not have been my choice to assign RfC status before getting to a resolution on that.
Also, it is possible for a JsonbValue to hold a timestamp (as a result of a jsonpath evaluation, I don't think that can happen any other way), and if such a jsonpath evaluation were to be the source expression of a cast to SQL timestamp, that situation seems exactly analogous to the other situations being optimized here and would require only a few more lines in the exact pattern here introduced. While that could be called out of scope when this patch's title refers to "numeric field" specifically, it might be worth considering for completeness. The patch does, after all, handle boolean already, as well as numeric.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-11-01 02:17 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
@ 2023-11-06 03:26 ` [email protected]
2024-01-02 08:18 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: [email protected] @ 2023-11-06 03:26 UTC (permalink / raw)
To: Chapman Flack <[email protected]>; +Cc: Andy Fan <[email protected]>; [email protected]
Chapman Flack <[email protected]> writes:
(This is Andy Fan and I just switch to my new email address).
Hi Chap,
Thanks for alway keep an eye on this!
> Adding this comment via the CF app so it isn't lost, while an
> improperly-interpreted-DKIM-headers issue is still preventing me from
> mailing directly to -hackers.
>
> It was my view that the patch was getting close by the end of the last
> commitfest, but still contained a bit of a logic wart made necessary by
> a questionable choice of error message wording, such that in my view it
> would be better to determine whether a different error message would
> better conform to ISO SQL in the first place, and obviate the need for
> the logic wart.
>
> There seemed to be some progress possible on that when petere had time
> to weigh in on the standard shortly after the last CF ended.
>
> So, it would not have been my choice to assign RfC status before
> getting to a resolution on that.
I agree with this.
>
> Also, it is possible for a JsonbValue to hold a timestamp (as a result
> of a jsonpath evaluation, I don't think that can happen any other
> way),
I believe this is where our disagreement lies.
CREATE TABLE employees (
id serial PRIMARY KEY,
data jsonb
);
INSERT INTO employees (data) VALUES (
'{
"employees":[
{
"firstName":"John",
"lastName":"Doe",
"hireDate":"2022-01-01T09:00:00Z",
"age": 30
},
{
"firstName":"Jane",
"lastName":"Smith",
"hireDate":"2022-02-01T10:00:00Z",
"age": 25
}
]
}'
);
select
jsonb_path_query_tz(data, '$.employees[*] ? (@.hireDate >=
"2022-02-01T00:00:00Z" && @.hireDate < "2022-03-01T00:00:00Z")')
from employees;
select jsonb_path_query_tz(data, '$.employees[*].hireDate ? (@ >=
"2022-02-01T00:00:00Z" && @ < "2022-03-01T00:00:00Z")') from employees;
select pg_typeof(jsonb_path_query_tz(data, '$.employees[*].hireDate ? (@
>= "2022-02-01T00:00:00Z" && @ < "2022-03-01T00:00:00Z")')) from
employees;
select jsonb_path_query_tz(data, '$.employees[*].hireDate ? (@
>= "2022-02-01T00:00:00Z" && @ < "2022-03-01T00:00:00Z")')::timestamp
from employees;
select jsonb_path_query_tz(data, '$.employees[*].hireDate ? (@
>= "2022-02-01T00:00:00Z" && @ < "2022-03-01T00:00:00Z")')::timestamptz
from employees;
I tried all of the above queires and can't find a place where this
optimization would apply. am I miss something?
> and if such a jsonpath evaluation were to be the source expression of a
> cast to SQL timestamp, that situation seems exactly analogous to the
> other situations being optimized here and would require only a few more
> lines in the exact pattern here introduced.
Could you provide an example of this?
> While that could be called
> out of scope when this patch's title refers to "numeric field"
> specifically, it might be worth considering for completeness. The patch
> does, after all, handle boolean already, as well as numeric.
I'd never arugment for this, at this point at least.
v15 is provides without any fundamental changes. Just rebase to the
lastest code and prepared a better commit message.
--
Best Regards
Andy Fan
Attachments:
[text/x-diff] v15-0001-Improve-the-performance-of-Jsonb-extraction.patch (38.6K, 2-v15-0001-Improve-the-performance-of-Jsonb-extraction.patch)
download | inline diff:
From 4d53fda4974fa18827350a5f42482f0eec29d6ba Mon Sep 17 00:00:00 2001
From: "yizhi.fzh" <[email protected]>
Date: Mon, 6 Nov 2023 11:09:14 +0800
Subject: [PATCH v15 1/1] Improve the performance of Jsonb extraction.
In the past, when we needed to extract a numeric value from a field in a
JSONB object, even though the JSONB object already contained a binary
matching numeric type, we would first find the corresponding JsonbValue,
then convert the JsonbValue to Jsonb, and finally use the cast system to
convert the Jsonb to a Numeric-like data type. This approach was very
inefficient in terms of performance.
In the current patch, if we encounter a function or operator that needs
to extract a JSONB field and cast it to a numeric-like data type, the
request will be automatically converted into extracting the field as a
JsonbValue data type from the JSONB field, then, convert the JsonbValue
to a numeric data type. If necessary, the final conversion from the
numeric data type to another numeric-like data type is done through the
casting system. This series of conversions is implemented through the
planner support function. By utilizing these methods, the cumbersome
JSONB-related operations are avoided. Because the boolean type and
numeric type share certain similarities in their attributes, we have
implemented the same optimization approach for both.
At the implementation level, considering that we have multiple operators
and various target data types, and to avoid an excessive number of
functions, we have deconstructed the two steps mentioned earlier into
two categories of functions. The first category of functions extracts
the data as a JsonbValue type, while the second category of functions
converts the JsonbValue type into the desired data type. In specific
scenarios, we utilize planner support functions to automatically
assemble these functions.
The optimized functions and operators includes:
1. jsonb_object_field / ->
2. jsonb_array_element / ->
3. jsonb_extract_path / #>
4. jsonb_path_query
5. jsonb_path_query_first
---
src/backend/utils/adt/jsonb.c | 166 ++++++++++++++++++++++++++
src/backend/utils/adt/jsonbsubs.c | 4 +-
src/backend/utils/adt/jsonfuncs.c | 118 ++++++++++++------
src/backend/utils/adt/jsonpath_exec.c | 32 +++--
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 46 +++++--
src/include/utils/jsonb.h | 11 +-
src/test/regress/expected/jsonb.out | 112 ++++++++++++++++-
src/test/regress/sql/jsonb.sql | 66 +++++++++-
src/tools/pgindent/typedefs.list | 1 +
10 files changed, 498 insertions(+), 60 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 6f445f5c2b..685f4e3e6b 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2039,6 +2043,168 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+
+/*
+ * jsonb_cast_support()
+ *
+ * Planner support function for casting a jsonb extraction to a numeric
+ * or bool data type. Instead of converting a jsonbvalue to jsonb, the new
+ * method will cast the jsonbvalue to the desired data type directly.
+ */
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = req->fcall;
+ FuncExpr *jsonb_start_func = NULL,
+ *jsonb_finish_func = NULL,
+ *final_func = NULL;
+ Node *input;
+ Oid new_func_id = InvalidOid;
+ List *args;
+ Oid input_func_id,
+ collid,
+ inputcollid;
+ bool retset = false,
+ variadic = false;
+
+ Assert(list_length(fexpr->args) == 1);
+ input = (Node *) linitial(fexpr->args);
+
+ if (IsA(input, OpExpr))
+ {
+ OpExpr *opExpr = castNode(OpExpr, input);
+
+ input_func_id = opExpr->opfuncid;
+ collid = opExpr->opcollid;
+ inputcollid = opExpr->inputcollid;
+ args = opExpr->args;
+ }
+ else if (IsA(input, FuncExpr))
+ {
+ FuncExpr *funcExpr = castNode(FuncExpr, input);
+
+ input_func_id = funcExpr->funcid;
+ collid = funcExpr->funccollid;
+ inputcollid = funcExpr->inputcollid;
+ args = funcExpr->args;
+ }
+ else
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(NULL);
+
+ /* build a function to return the JsonbValue directly. */
+ switch (input_func_id)
+ {
+ case F_JSONB_OBJECT_FIELD:
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ break;
+ case F_JSONB_ARRAY_ELEMENT:
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ break;
+ case F_JSONB_EXTRACT_PATH:
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+ variadic = true;
+ break;
+ case F_JSONB_PATH_QUERY:
+ new_func_id = F_JSONB_PATH_QUERY_START;
+ retset = true;
+ break;
+ case F_JSONB_PATH_QUERY_FIRST:
+ new_func_id = F_JSONB_PATH_QUERY_FIRST_START;
+ break;
+ default:
+ new_func_id = InvalidOid;
+ break;
+ }
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(NULL);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, args,
+ collid, inputcollid,
+ COERCE_EXPLICIT_CALL);
+ jsonb_start_func->funcretset = retset;
+ jsonb_start_func->funcvariadic = variadic;
+
+ /* relabel the first argument as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, -1,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ /* build the function to turn the JsonbValue into numeric */
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make2(jsonb_start_func,
+ makeConst(OIDOID,
+ -1,
+ InvalidOid,
+ sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false,
+ true)),
+ collid, inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ /*
+ * leverage the casting system to turn the numeric to
+ * desired type.
+ */
+ final_func = (FuncExpr *) coerce_type(NULL, (Node *) jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), collid,
+ inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(NULL);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *) PG_GETARG_POINTER(0);
+ Oid final_oid = PG_GETARG_OID(1);
+
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, format_type_be(final_oid));
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *) PG_GETARG_POINTER(0);
+
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonbsubs.c b/src/backend/utils/adt/jsonbsubs.c
index de0ae3604f..cb050c65ef 100644
--- a/src/backend/utils/adt/jsonbsubs.c
+++ b/src/backend/utils/adt/jsonbsubs.c
@@ -251,7 +251,7 @@ jsonb_subscript_fetch(ExprState *state,
workspace->index,
sbsrefstate->numupper,
op->resnull,
- false);
+ JsonbValue_AsJsonb);
}
/*
@@ -343,7 +343,7 @@ jsonb_subscript_fetch_old(ExprState *state,
sbsrefstate->upperindex,
sbsrefstate->numupper,
&sbsrefstate->prevnull,
- false);
+ JsonbValue_AsJsonb);
}
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index aa37c401e5..4e73c0f5c7 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -357,7 +357,7 @@ static JsonParseErrorType get_scalar(void *state, char *token, JsonTokenType tok
static Datum get_path_all(FunctionCallInfo fcinfo, bool as_text);
static text *get_worker(text *json, char **tpath, int *ipath, int npath,
bool normalize_results);
-static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text);
+static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target);
static text *JsonbValueAsText(JsonbValue *v);
/* semantic action functions for json_array_length */
@@ -492,6 +492,20 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+Datum
+jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target)
+{
+ switch (target)
+ {
+ case JsonbValue_AsJsonbValue:
+ PG_RETURN_POINTER(jbv);
+ case JsonbValue_AsJsonb:
+ PG_RETURN_JSONB_P(JsonbValueToJsonb(jbv));
+ case JsonbValue_AsText:
+ PG_RETURN_TEXT_P(JsonbValueAsText(jbv));
+ }
+ PG_RETURN_POINTER(NULL);
+}
/*
* pg_parse_json_or_errsave
@@ -847,13 +861,12 @@ json_object_field(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_object_field(PG_FUNCTION_ARGS)
+static Datum
+jsonb_object_field_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
text *key = PG_GETARG_TEXT_PP(1);
JsonbValue *v;
- JsonbValue vbuf;
if (!JB_ROOT_IS_OBJECT(jb))
PG_RETURN_NULL();
@@ -861,14 +874,26 @@ jsonb_object_field(PG_FUNCTION_ARGS)
v = getKeyJsonValueFromContainer(&jb->root,
VARDATA_ANY(key),
VARSIZE_ANY_EXHDR(key),
- &vbuf);
+ NULL);
if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
+ return jsonbvalue_covert(v, target);
PG_RETURN_NULL();
}
+Datum
+jsonb_object_field(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonbValue);
+}
+
Datum
json_object_field_text(PG_FUNCTION_ARGS)
{
@@ -922,8 +947,8 @@ json_array_element(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
-Datum
-jsonb_array_element(PG_FUNCTION_ARGS)
+static Datum
+jsonb_array_element_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
int element = PG_GETARG_INT32(1);
@@ -945,11 +970,23 @@ jsonb_array_element(PG_FUNCTION_ARGS)
v = getIthJsonbValueFromContainer(&jb->root, element);
if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
+ return jsonbvalue_covert(v, target);
PG_RETURN_NULL();
}
+Datum
+jsonb_array_element(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonbValue);
+}
+
Datum
json_array_element_text(PG_FUNCTION_ARGS)
{
@@ -1476,17 +1513,23 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, false);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonb);
}
Datum
jsonb_extract_path_text(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, true);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsText);
+}
+
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonbValue);
}
static Datum
-get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
+get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
@@ -1508,7 +1551,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
- res = jsonb_get_element(jb, pathtext, npath, &isnull, as_text);
+ res = jsonb_get_element(jb, pathtext, npath, &isnull, target);
if (isnull)
PG_RETURN_NULL();
@@ -1517,7 +1560,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
}
Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, JsonbValueTarget target)
{
JsonbContainer *container = &jb->root;
JsonbValue *jbvp = NULL;
@@ -1550,16 +1593,26 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
*/
if (npath <= 0 && jbvp == NULL)
{
- if (as_text)
- {
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
+ switch (target)
{
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsText:
+ return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
+ container,
+ VARSIZE(jb))));
+ /* not text mode - just hand back the jsonb */
+ case JsonbValue_AsJsonb:
+ PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsJsonbValue:
+ {
+ JsonbValue *jbv = NULL;
+
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ PG_RETURN_POINTER(getIthJsonbValueFromContainer(container, 0));
+
+ jbv = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, jbv);
+ PG_RETURN_POINTER(jbv);
+ }
}
}
@@ -1645,23 +1698,14 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
}
}
- if (as_text)
- {
- if (jbvp->type == jbvNull)
- {
- *isnull = true;
- return PointerGetDatum(NULL);
- }
- return PointerGetDatum(JsonbValueAsText(jbvp));
- }
- else
+ if (target == JsonbValue_AsText && jbvp->type == jbvNull)
{
- Jsonb *res = JsonbValueToJsonb(jbvp);
-
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(res);
+ *isnull = true;
+ return PointerGetDatum(NULL);
}
+
+ return jsonbvalue_covert(jbvp, target);
}
Datum
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 2d0599b4aa..03045c7ba9 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -393,7 +393,7 @@ jsonb_path_match_opr(PG_FUNCTION_ARGS)
* rowset.
*/
static Datum
-jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
FuncCallContext *funcctx;
List *found;
@@ -435,19 +435,25 @@ jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
v = lfirst(c);
funcctx->user_fctx = list_delete_first(found);
- SRF_RETURN_NEXT(funcctx, JsonbPGetDatum(JsonbValueToJsonb(v)));
+ SRF_RETURN_NEXT(funcctx, jsonbvalue_covert(v, target));
}
Datum
jsonb_path_query(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, false);
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, true);
+ return jsonb_path_query_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/*
@@ -487,7 +493,7 @@ jsonb_path_query_array_tz(PG_FUNCTION_ARGS)
* item. If there are no items, NULL returned.
*/
static Datum
-jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
JsonPath *jp = PG_GETARG_JSONPATH_P(1);
@@ -498,7 +504,11 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
(void) executeJsonPath(jp, vars, jb, !silent, &found, tz);
if (JsonValueListLength(&found) >= 1)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(JsonValueListHead(&found)));
+ {
+ JsonbValue *jbv = JsonValueListHead(&found);
+
+ return jsonbvalue_covert(jbv, target);
+ }
else
PG_RETURN_NULL();
}
@@ -506,13 +516,19 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
Datum
jsonb_path_query_first(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, false);
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_first_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, true);
+ return jsonb_path_query_first_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_first_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/********************Execute functions for JsonPath**************************/
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index bee21befda..802c6f1925 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202310301
+#define CATALOG_VERSION_NO 202311061
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 091f7e343c..6934d0894d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4590,25 +4590,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9951,6 +9951,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552', descr => 'extract jsonbvalue from jsonb for the given field',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '3813', descr => 'extract josnbvalue from jsonb array for the given index',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4551', descr => 'extract jsonbvalue from jsonb for the given paths',
+ proname => 'jsonb_extract_path_start', provariadic => 'text', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{internal,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553', descr => 'convert a jsonbvalue to numeric',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal oid', proargnames => '{from_jsonvalue,target_oid}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554', descr => 'convert a jsonbvalue to boolean',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -10143,6 +10167,10 @@
proname => 'jsonb_path_query', prorows => '1000', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query' },
+{ oid => '4557', descr => 'jsonpath query as jsonbvalue',
+ proname => 'jsonb_path_query_start', prorows => '1000', proretset => 't',
+ prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_start' },
{ oid => '4007', descr => 'jsonpath query wrapped into array',
proname => 'jsonb_path_query_array', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
@@ -10151,6 +10179,10 @@
proname => 'jsonb_path_query_first', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query_first' },
+{ oid => '4555', descr => 'jsonpath query first item as jsonbvalue',
+ proname => 'jsonb_path_query_first_start', prorettype => 'internal',
+ proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_first_start' },
{ oid => '4009', descr => 'jsonpath match',
proname => 'jsonb_path_match', prorettype => 'bool',
proargtypes => 'jsonb jsonpath jsonb bool', prosrc => 'jsonb_path_match' },
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index addc9b608e..5e0014f040 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -294,6 +294,13 @@ struct JsonbValue
} val;
};
+typedef enum JsonbValueTarget
+{
+ JsonbValue_AsJsonbValue,
+ JsonbValue_AsJsonb,
+ JsonbValue_AsText
+} JsonbValueTarget;
+
#define IsAJsonbScalar(jsonbval) (((jsonbval)->type >= jbvNull && \
(jsonbval)->type <= jbvBool) || \
(jsonbval)->type == jbvDatetime)
@@ -428,12 +435,12 @@ extern const char *JsonbTypeName(JsonbValue *val);
extern Datum jsonb_set_element(Jsonb *jb, Datum *path, int path_len,
JsonbValue *newval);
extern Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath,
- bool *isnull, bool as_text);
+ bool *isnull, JsonbValueTarget target);
extern bool to_jsonb_is_immutable(Oid typoid);
extern Datum jsonb_build_object_worker(int nargs, const Datum *args, const bool *nulls,
const Oid *types, bool absent_on_null,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, const Datum *args, const bool *nulls,
const Oid *types, bool absent_on_null);
-
+extern Datum jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index 4a16d0dbaf..c6af7a59cc 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,114 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '1700'::oid), (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '23'::oid))::integer, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '20'::oid))::bigint, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '700'::oid))::real, (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '701'::oid))::double precision, jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 0), '1700'::oid), jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 10), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f1}'::text[]), '1700'::oid), jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f2}'::text[]), '1700'::oid), jsonb_finish_bool(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Result
+ Output: (jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), jsonb_path_query_first(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), (jsonb_finish_numeric((jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), '21'::oid))::smallint, (jsonb_finish_numeric(jsonb_path_query_first_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), '21'::oid))::smallint
+ -> ProjectSet
+ Output: jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), test_json
+ -> Seq Scan on pg_temp.test_jsonb
+ Output: json_type, test_json
+ Filter: (test_jsonb.json_type = 'object'::text)
+(7 rows)
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ jsonb_path_query | jsonb_path_query_first | jsonb_path_query | jsonb_path_query_first
+------------------+------------------------+------------------+------------------------
+ 2 | 2 | 2 | 2
+ 3 | 2 | 3 | 2
+ 4 | 2 | 4 | 2
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field1'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type integer
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +692,9 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+ field8
+(8 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index e4b7cdf703..500d04936d 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,72 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}, "field7": true, "field8": [1,2,3,4,5]}');
+
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json->'field5' -> 0)::numeric,
+(test_json->'field5' -> 10)::numeric,
+(test_json#>'{"field6", "f1"}')::numeric,
+(test_json#>'{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 87c1aee379..b0f66589e8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1256,6 +1256,7 @@ JsonArrayAgg
JsonArrayConstructor
JsonArrayQueryConstructor
JsonBaseObjectInfo
+JsonbValueTarget
JsonConstructorExpr
JsonConstructorExprState
JsonConstructorType
--
2.34.1
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-11-01 02:17 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-11-06 03:26 ` Re: Extract numeric filed in JSONB more effectively [email protected]
@ 2024-01-02 08:18 ` jian he <[email protected]>
2024-01-07 07:17 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: jian he @ 2024-01-02 08:18 UTC (permalink / raw)
To: [email protected]; +Cc: Chapman Flack <[email protected]>; [email protected]
hi.
you don't need to change src/include/catalog/catversion.h
as mentioned in https://wiki.postgresql.org/wiki/Committing_checklist
Otherwise, cfbot will fail many times.
+typedef enum JsonbValueTarget
+{
+ JsonbValue_AsJsonbValue,
+ JsonbValue_AsJsonb,
+ JsonbValue_AsText
+} JsonbValueTarget;
change to
+typedef enum JsonbValueTarget
+{
+ JsonbValue_AsJsonbValue,
+ JsonbValue_AsJsonb,
+ JsonbValue_AsText,
+} JsonbValueTarget;
currently cannot do `git apply`.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-11-01 02:17 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-11-06 03:26 ` Re: Extract numeric filed in JSONB more effectively [email protected]
2024-01-02 08:18 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
@ 2024-01-07 07:17 ` Andy Fan <[email protected]>
2024-01-08 00:00 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andy Fan @ 2024-01-07 07:17 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Chapman Flack <[email protected]>; [email protected]
Hi,
> hi.
> you don't need to change src/include/catalog/catversion.h
> as mentioned in https://wiki.postgresql.org/wiki/Committing_checklist
> Otherwise, cfbot will fail many times.
Thanks for the wiki.
I checked the wiki and search "catversion", the only message I got is:
"Consider the need for a catversion bump."
How could this be explained as "no need to change ../catversion.h"?
>
> +typedef enum JsonbValueTarget
> +{
> + JsonbValue_AsJsonbValue,
> + JsonbValue_AsJsonb,
> + JsonbValue_AsText
> +} JsonbValueTarget;
>
> change to
>
> +typedef enum JsonbValueTarget
> +{
> + JsonbValue_AsJsonbValue,
> + JsonbValue_AsJsonb,
> + JsonbValue_AsText,
> +} JsonbValueTarget;
>
> currently cannot do `git apply`.
OK, I guess it's something about whitespaces, my git-commit hook has
been configured to capture this during commit. After we reach an
agreement about the 'catversion.h' stuff, the next version of patch
should fix this issue.
--
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Extract numeric filed in JSONB more effectively
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-17 20:30 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 01:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 02:55 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 07:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-18 18:50 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-18 19:08 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-21 01:31 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-21 03:19 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-22 03:14 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 05:54 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-22 12:16 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-26 22:28 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-30 04:47 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-08-30 13:47 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-08-31 09:10 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-01 03:09 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-02 01:25 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-09-04 11:43 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-04 14:35 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-13 21:18 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-09-14 06:41 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-09-15 01:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-10-05 06:53 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
2023-11-01 02:17 ` Re: Extract numeric filed in JSONB more effectively Chapman Flack <[email protected]>
2023-11-06 03:26 ` Re: Extract numeric filed in JSONB more effectively [email protected]
2024-01-02 08:18 ` Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2024-01-07 07:17 ` Re: Extract numeric filed in JSONB more effectively Andy Fan <[email protected]>
@ 2024-01-08 00:00 ` jian he <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: jian he @ 2024-01-08 00:00 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: Chapman Flack <[email protected]>; [email protected]
On Sun, Jan 7, 2024 at 3:26 PM Andy Fan <[email protected]> wrote:
>
>
> Hi,
>
> > hi.
> > you don't need to change src/include/catalog/catversion.h
> > as mentioned in https://wiki.postgresql.org/wiki/Committing_checklist
> > Otherwise, cfbot will fail many times.
>
> Thanks for the wiki.
>
> I checked the wiki and search "catversion", the only message I got is:
>
> "Consider the need for a catversion bump."
>
> How could this be explained as "no need to change ../catversion.h"?
that means catversion.h changes is the committer's responsibility, IMHO.
IMHO, main reason is every time the catversion.h change, cfbot
http://cfbot.cputube.org will fail.
one patch took very long time to be committable.
you don't need update your patch for the every catversion.h changes.
> >
> > +typedef enum JsonbValueTarget
> > +{
> > + JsonbValue_AsJsonbValue,
> > + JsonbValue_AsJsonb,
> > + JsonbValue_AsText
> > +} JsonbValueTarget;
> >
> > change to
> >
> > +typedef enum JsonbValueTarget
> > +{
> > + JsonbValue_AsJsonbValue,
> > + JsonbValue_AsJsonb,
> > + JsonbValue_AsText,
> > +} JsonbValueTarget;
> >
reason: https://git.postgresql.org/cgit/postgresql.git/commit/?id=611806cd726fc92989ac918eac48fd8d684869c7
> > currently cannot do `git apply`.
>
> OK, I guess it's something about whitespaces, my git-commit hook has
> been configured to capture this during commit. After we reach an
> agreement about the 'catversion.h' stuff, the next version of patch
> should fix this issue.
Anyway, I made the following change:
remove catversion.h changes.
refactored the tests. Some of the explain(costs off, verbose) output
is very very long.
it's unreadable on the web browser. so I cut them into small pieces.
resolve duplicate OID issues.
slight refactored jsonbvalue_covert function, for the switch
statement, add a default branch.
see file v16-0001-Improve-the-performance-of-Jsonb-extraction.patch
you made a lot of changes, that might not be easy to get committed, i think.
Maybe we can split the patch into several pieces.
The first part is the original idea that: pattern: (jsonb(object) ->
'key')::numerica_data_type can be optimized.
The second part: is other cases where cast jsonb to scalar data type
can also be optimized.
So, I refactor your patch. only have optimized casts for:
(jsonb(object) -> 'key')::numerica_data_type.
We can optimize more cast cases, but IMHO,
make it as minimal as possible, easier to review, easier to understand.
If people think this performance gain is good, then later we can add
more on top of it.
summary: 2 files attached.
v16-0001-Improve-the-performance-of-Jsonb-extraction.patch
refactored of your patch, that covers all the cast optimization cases,
this file will run the CI test.
v1-0001-Improve-performance-of-Jsonb-extract-via-key-and-c.no-cfbot
this one also based on your patch. but as a minimum patch to optimize
(jsonb(object) -> 'key')::numerica_data_type case only. (this one will
not run CI test).
Attachments:
[text/x-patch] v16-0001-Improve-the-performance-of-Jsonb-extraction.patch (42.5K, 2-v16-0001-Improve-the-performance-of-Jsonb-extraction.patch)
download | inline diff:
From 5ac139d3eb213beaa53e6cd39c25b2ecb334ba56 Mon Sep 17 00:00:00 2001
From: jian he <[email protected]>
Date: Sun, 7 Jan 2024 14:14:11 +0800
Subject: [PATCH v16 1/1] Improve the performance of Jsonb extraction.
In the past, when we needed to extract a numeric value from a field in a
JSONB object, even though the JSONB object already contained a binary
matching numeric type, we would first find the corresponding JsonbValue,
then convert the JsonbValue to Jsonb, and finally use the cast system to
convert the Jsonb to a Numeric-like data type. This approach was very
inefficient in terms of performance.
In the current patch, if we encounter a function or operator that needs
to extract a JSONB field and cast it to a numeric-like data type, the
request will be automatically converted into extracting the field as a
JsonbValue data type from the JSONB field, then, convert the JsonbValue
to a numeric data type. If necessary, the final conversion from the
numeric data type to another numeric-like data type is done through the
casting system. This series of conversions is implemented through the
planner support function. By utilizing these methods, the cumbersome
JSONB-related operations are avoided. Because the boolean type and
numeric type share certain similarities in their attributes, we have
implemented the same optimization approach for both.
At the implementation level, considering that we have multiple operators
and various target data types, and to avoid an excessive number of
functions, we have deconstructed the two steps mentioned earlier into
two categories of functions. The first category of functions extracts
the data as a JsonbValue type, while the second category of functions
converts the JsonbValue type into the desired data type. In specific
scenarios, we utilize planner support functions to automatically
assemble these functions.
---
src/backend/utils/adt/jsonb.c | 166 ++++++++++++++++++
src/backend/utils/adt/jsonbsubs.c | 4 +-
src/backend/utils/adt/jsonfuncs.c | 180 ++++++++++++-------
src/backend/utils/adt/jsonpath_exec.c | 32 +++-
src/include/catalog/pg_proc.dat | 46 ++++-
src/include/utils/jsonb.h | 11 +-
src/test/regress/expected/jsonb.out | 240 +++++++++++++++++++++++++-
src/test/regress/sql/jsonb.sql | 116 ++++++++++++-
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 707 insertions(+), 89 deletions(-)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index c10b3fbe..b5a09b61 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -17,11 +17,15 @@
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "nodes/makefuncs.h"
+#include "nodes/supportnodes.h"
+#include "parser/parse_coerce.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
@@ -2039,6 +2043,168 @@ cannotCastJsonbValue(enum jbvType type, const char *sqltype)
elog(ERROR, "unknown jsonb type: %d", (int) type);
}
+
+/*
+ * jsonb_cast_support()
+ *
+ * Planner support function for casting a jsonb extraction to a numeric
+ * or bool data type. Instead of converting a jsonbvalue to jsonb, the new
+ * method will cast the jsonbvalue to the desired data type directly.
+ */
+Datum
+jsonb_cast_support(PG_FUNCTION_ARGS)
+{
+ Node *rawreq = (Node *) PG_GETARG_POINTER(0);
+
+ if (IsA(rawreq, SupportRequestSimplify))
+ {
+ SupportRequestSimplify *req = (SupportRequestSimplify *) rawreq;
+ FuncExpr *fexpr = req->fcall;
+ FuncExpr *jsonb_start_func = NULL,
+ *jsonb_finish_func = NULL,
+ *final_func = NULL;
+ Node *input;
+ Oid new_func_id = InvalidOid;
+ List *args;
+ Oid input_func_id,
+ collid,
+ inputcollid;
+ bool retset = false,
+ variadic = false;
+
+ Assert(list_length(fexpr->args) == 1);
+ input = (Node *) linitial(fexpr->args);
+
+ if (IsA(input, OpExpr))
+ {
+ OpExpr *opExpr = castNode(OpExpr, input);
+
+ input_func_id = opExpr->opfuncid;
+ collid = opExpr->opcollid;
+ inputcollid = opExpr->inputcollid;
+ args = opExpr->args;
+ }
+ else if (IsA(input, FuncExpr))
+ {
+ FuncExpr *funcExpr = castNode(FuncExpr, input);
+
+ input_func_id = funcExpr->funcid;
+ collid = funcExpr->funccollid;
+ inputcollid = funcExpr->inputcollid;
+ args = funcExpr->args;
+ }
+ else
+ /* not the desired pattern. */
+ PG_RETURN_POINTER(NULL);
+
+ /* build a function to return the JsonbValue directly. */
+ switch (input_func_id)
+ {
+ case F_JSONB_OBJECT_FIELD:
+ new_func_id = F_JSONB_OBJECT_FIELD_START;
+ break;
+ case F_JSONB_ARRAY_ELEMENT:
+ new_func_id = F_JSONB_ARRAY_ELEMENT_START;
+ break;
+ case F_JSONB_EXTRACT_PATH:
+ new_func_id = F_JSONB_EXTRACT_PATH_START;
+ variadic = true;
+ break;
+ case F_JSONB_PATH_QUERY:
+ new_func_id = F_JSONB_PATH_QUERY_START;
+ retset = true;
+ break;
+ case F_JSONB_PATH_QUERY_FIRST:
+ new_func_id = F_JSONB_PATH_QUERY_FIRST_START;
+ break;
+ default:
+ new_func_id = InvalidOid;
+ break;
+ }
+
+ if (!OidIsValid(new_func_id))
+ PG_RETURN_POINTER(NULL);
+
+ jsonb_start_func = makeFuncExpr(new_func_id, INTERNALOID, args,
+ collid, inputcollid,
+ COERCE_EXPLICIT_CALL);
+ jsonb_start_func->funcretset = retset;
+ jsonb_start_func->funcvariadic = variadic;
+
+ /* relabel the first argument as 'internal'. */
+ linitial(jsonb_start_func->args) = makeRelabelType(linitial(jsonb_start_func->args),
+ INTERNALOID, -1,
+ InvalidOid,
+ COERCE_IMPLICIT_CAST);
+ switch (fexpr->funcresulttype)
+ {
+ case INT2OID:
+ case INT4OID:
+ case INT8OID:
+ case FLOAT4OID:
+ case FLOAT8OID:
+ case NUMERICOID:
+ /* build the function to turn the JsonbValue into numeric */
+ jsonb_finish_func = makeFuncExpr(F_JSONB_FINISH_NUMERIC, NUMERICOID,
+ list_make2(jsonb_start_func,
+ makeConst(OIDOID,
+ -1,
+ InvalidOid,
+ sizeof(Oid),
+ ObjectIdGetDatum(fexpr->funcresulttype),
+ false,
+ true)),
+ collid, inputcollid, COERCE_EXPLICIT_CALL);
+
+ if (fexpr->funcresulttype != NUMERICOID)
+ {
+ /*
+ * leverage the casting system to turn the numeric to
+ * desired type.
+ */
+ final_func = (FuncExpr *) coerce_type(NULL, (Node *) jsonb_finish_func, NUMERICOID,
+ fexpr->funcresulttype, 0, COERCION_EXPLICIT,
+ COERCE_EXPLICIT_CAST, fexpr->location);
+ }
+ else
+ final_func = jsonb_finish_func;
+
+ PG_RETURN_POINTER(final_func);
+ case BOOLOID:
+ final_func = makeFuncExpr(F_JSONB_FINISH_BOOL, BOOLOID,
+ list_make1(jsonb_start_func), collid,
+ inputcollid, COERCE_EXPLICIT_CALL);
+ PG_RETURN_POINTER(final_func);
+ default:
+ PG_RETURN_POINTER(NULL);
+ }
+ }
+
+ PG_RETURN_POINTER(NULL);
+}
+
+
+Datum
+jsonb_finish_numeric(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *) PG_GETARG_POINTER(0);
+ Oid final_oid = PG_GETARG_OID(1);
+
+ if (v->type != jbvNumeric)
+ cannotCastJsonbValue(v->type, format_type_be(final_oid));
+ PG_RETURN_NUMERIC(v->val.numeric);
+}
+
+Datum
+jsonb_finish_bool(PG_FUNCTION_ARGS)
+{
+ JsonbValue *v = (JsonbValue *) PG_GETARG_POINTER(0);
+
+ if (v->type != jbvBool)
+ cannotCastJsonbValue(v->type, "boolean");
+ PG_RETURN_BOOL(v->val.boolean);
+}
+
Datum
jsonb_bool(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/jsonbsubs.c b/src/backend/utils/adt/jsonbsubs.c
index 79c5d16f..37be80d1 100644
--- a/src/backend/utils/adt/jsonbsubs.c
+++ b/src/backend/utils/adt/jsonbsubs.c
@@ -251,7 +251,7 @@ jsonb_subscript_fetch(ExprState *state,
workspace->index,
sbsrefstate->numupper,
op->resnull,
- false);
+ JsonbValue_AsJsonb);
}
/*
@@ -343,7 +343,7 @@ jsonb_subscript_fetch_old(ExprState *state,
sbsrefstate->upperindex,
sbsrefstate->numupper,
&sbsrefstate->prevnull,
- false);
+ JsonbValue_AsJsonb);
}
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index caaafb72..baeefabc 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -357,7 +357,7 @@ static JsonParseErrorType get_scalar(void *state, char *token, JsonTokenType tok
static Datum get_path_all(FunctionCallInfo fcinfo, bool as_text);
static text *get_worker(text *json, char **tpath, int *ipath, int npath,
bool normalize_results);
-static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text);
+static Datum get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target);
static text *JsonbValueAsText(JsonbValue *v);
/* semantic action functions for json_array_length */
@@ -492,6 +492,22 @@ static JsonParseErrorType transform_string_values_object_field_start(void *state
static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull);
static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype);
+Datum
+jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target)
+{
+ switch (target)
+ {
+ case JsonbValue_AsJsonbValue:
+ PG_RETURN_POINTER(jbv);
+ case JsonbValue_AsJsonb:
+ PG_RETURN_JSONB_P(JsonbValueToJsonb(jbv));
+ case JsonbValue_AsText:
+ PG_RETURN_TEXT_P(JsonbValueAsText(jbv));
+ default:
+ elog(ERROR, "invalid jsonbvalue cast target type");
+ }
+ pg_unreachable();
+}
/*
* pg_parse_json_or_errsave
@@ -847,26 +863,37 @@ json_object_field(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
+static Datum
+jsonb_object_field_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ text *key = PG_GETARG_TEXT_PP(1);
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_OBJECT(jb))
+ PG_RETURN_NULL();
+
+ v = getKeyJsonValueFromContainer(&jb->root,
+ VARDATA_ANY(key),
+ VARSIZE_ANY_EXHDR(key),
+ NULL);
+
+ if (v != NULL)
+ return jsonbvalue_covert(v, target);
+
+ PG_RETURN_NULL();
+}
+
Datum
jsonb_object_field(PG_FUNCTION_ARGS)
{
- Jsonb *jb = PG_GETARG_JSONB_P(0);
- text *key = PG_GETARG_TEXT_PP(1);
- JsonbValue *v;
- JsonbValue vbuf;
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonb);
+}
- if (!JB_ROOT_IS_OBJECT(jb))
- PG_RETURN_NULL();
-
- v = getKeyJsonValueFromContainer(&jb->root,
- VARDATA_ANY(key),
- VARSIZE_ANY_EXHDR(key),
- &vbuf);
-
- if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
-
- PG_RETURN_NULL();
+Datum
+jsonb_object_field_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_object_field_internal(fcinfo, JsonbValue_AsJsonbValue);
}
Datum
@@ -922,32 +949,44 @@ json_array_element(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
+static Datum
+jsonb_array_element_internal(FunctionCallInfo fcinfo, JsonbValueTarget target)
+{
+ Jsonb *jb = PG_GETARG_JSONB_P(0);
+ int element = PG_GETARG_INT32(1);
+ JsonbValue *v;
+
+ if (!JB_ROOT_IS_ARRAY(jb))
+ PG_RETURN_NULL();
+
+ /* Handle negative subscript */
+ if (element < 0)
+ {
+ uint32 nelements = JB_ROOT_COUNT(jb);
+
+ if (-element > nelements)
+ PG_RETURN_NULL();
+ else
+ element += nelements;
+ }
+
+ v = getIthJsonbValueFromContainer(&jb->root, element);
+ if (v != NULL)
+ return jsonbvalue_covert(v, target);
+
+ PG_RETURN_NULL();
+}
+
Datum
jsonb_array_element(PG_FUNCTION_ARGS)
{
- Jsonb *jb = PG_GETARG_JSONB_P(0);
- int element = PG_GETARG_INT32(1);
- JsonbValue *v;
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonb);
+}
- if (!JB_ROOT_IS_ARRAY(jb))
- PG_RETURN_NULL();
-
- /* Handle negative subscript */
- if (element < 0)
- {
- uint32 nelements = JB_ROOT_COUNT(jb);
-
- if (-element > nelements)
- PG_RETURN_NULL();
- else
- element += nelements;
- }
-
- v = getIthJsonbValueFromContainer(&jb->root, element);
- if (v != NULL)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(v));
-
- PG_RETURN_NULL();
+Datum
+jsonb_array_element_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_array_element_internal(fcinfo, JsonbValue_AsJsonbValue);
}
Datum
@@ -1476,17 +1515,23 @@ get_scalar(void *state, char *token, JsonTokenType tokentype)
Datum
jsonb_extract_path(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, false);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonb);
}
Datum
jsonb_extract_path_text(PG_FUNCTION_ARGS)
{
- return get_jsonb_path_all(fcinfo, true);
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsText);
+}
+
+Datum
+jsonb_extract_path_start(PG_FUNCTION_ARGS)
+{
+ return get_jsonb_path_all(fcinfo, JsonbValue_AsJsonbValue);
}
static Datum
-get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
+get_jsonb_path_all(FunctionCallInfo fcinfo, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
ArrayType *path = PG_GETARG_ARRAYTYPE_P(1);
@@ -1508,7 +1553,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
deconstruct_array_builtin(path, TEXTOID, &pathtext, &pathnulls, &npath);
- res = jsonb_get_element(jb, pathtext, npath, &isnull, as_text);
+ res = jsonb_get_element(jb, pathtext, npath, &isnull, target);
if (isnull)
PG_RETURN_NULL();
@@ -1517,7 +1562,7 @@ get_jsonb_path_all(FunctionCallInfo fcinfo, bool as_text)
}
Datum
-jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
+jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, JsonbValueTarget target)
{
JsonbContainer *container = &jb->root;
JsonbValue *jbvp = NULL;
@@ -1550,16 +1595,26 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
*/
if (npath <= 0 && jbvp == NULL)
{
- if (as_text)
+ switch (target)
{
- return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
- container,
- VARSIZE(jb))));
- }
- else
- {
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsText:
+ return PointerGetDatum(cstring_to_text(JsonbToCString(NULL,
+ container,
+ VARSIZE(jb))));
+ /* not text mode - just hand back the jsonb */
+ case JsonbValue_AsJsonb:
+ PG_RETURN_JSONB_P(jb);
+ case JsonbValue_AsJsonbValue:
+ {
+ JsonbValue *jbv = NULL;
+
+ if (JB_ROOT_IS_ARRAY(jb) && JB_ROOT_IS_SCALAR(jb))
+ PG_RETURN_POINTER(getIthJsonbValueFromContainer(container, 0));
+
+ jbv = palloc0(sizeof(JsonbValue));
+ JsonbToJsonbValue(jb, jbv);
+ PG_RETURN_POINTER(jbv);
+ }
}
}
@@ -1645,23 +1700,14 @@ jsonb_get_element(Jsonb *jb, Datum *path, int npath, bool *isnull, bool as_text)
}
}
- if (as_text)
- {
- if (jbvp->type == jbvNull)
- {
- *isnull = true;
- return PointerGetDatum(NULL);
- }
- return PointerGetDatum(JsonbValueAsText(jbvp));
- }
- else
+ if (target == JsonbValue_AsText && jbvp->type == jbvNull)
{
- Jsonb *res = JsonbValueToJsonb(jbvp);
-
- /* not text mode - just hand back the jsonb */
- PG_RETURN_JSONB_P(res);
+ *isnull = true;
+ return PointerGetDatum(NULL);
}
+
+ return jsonbvalue_covert(jbvp, target);
}
Datum
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index ac16f5c8..b121a6a3 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -393,7 +393,7 @@ jsonb_path_match_opr(PG_FUNCTION_ARGS)
* rowset.
*/
static Datum
-jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
FuncCallContext *funcctx;
List *found;
@@ -435,19 +435,25 @@ jsonb_path_query_internal(FunctionCallInfo fcinfo, bool tz)
v = lfirst(c);
funcctx->user_fctx = list_delete_first(found);
- SRF_RETURN_NEXT(funcctx, JsonbPGetDatum(JsonbValueToJsonb(v)));
+ SRF_RETURN_NEXT(funcctx, jsonbvalue_covert(v, target));
}
Datum
jsonb_path_query(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, false);
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_internal(fcinfo, true);
+ return jsonb_path_query_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/*
@@ -487,7 +493,7 @@ jsonb_path_query_array_tz(PG_FUNCTION_ARGS)
* item. If there are no items, NULL returned.
*/
static Datum
-jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
+jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz, JsonbValueTarget target)
{
Jsonb *jb = PG_GETARG_JSONB_P(0);
JsonPath *jp = PG_GETARG_JSONPATH_P(1);
@@ -498,7 +504,11 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
(void) executeJsonPath(jp, vars, jb, !silent, &found, tz);
if (JsonValueListLength(&found) >= 1)
- PG_RETURN_JSONB_P(JsonbValueToJsonb(JsonValueListHead(&found)));
+ {
+ JsonbValue *jbv = JsonValueListHead(&found);
+
+ return jsonbvalue_covert(jbv, target);
+ }
else
PG_RETURN_NULL();
}
@@ -506,13 +516,19 @@ jsonb_path_query_first_internal(FunctionCallInfo fcinfo, bool tz)
Datum
jsonb_path_query_first(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, false);
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonb);
}
Datum
jsonb_path_query_first_tz(PG_FUNCTION_ARGS)
{
- return jsonb_path_query_first_internal(fcinfo, true);
+ return jsonb_path_query_first_internal(fcinfo, true, JsonbValue_AsJsonb);
+}
+
+Datum
+jsonb_path_query_first_start(PG_FUNCTION_ARGS)
+{
+ return jsonb_path_query_first_internal(fcinfo, false, JsonbValue_AsJsonbValue);
}
/********************Execute functions for JsonPath**************************/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 79793927..3922ac4d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -4590,25 +4590,25 @@
proname => 'pg_lsn', prorettype => 'pg_lsn', proargtypes => 'numeric',
prosrc => 'numeric_pg_lsn' },
-{ oid => '3556', descr => 'convert jsonb to boolean',
+{ oid => '3556', descr => 'convert jsonb to boolean', prosupport => 'jsonb_cast_support',
proname => 'bool', prorettype => 'bool', proargtypes => 'jsonb',
prosrc => 'jsonb_bool' },
{ oid => '3449', descr => 'convert jsonb to numeric',
- proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb',
+ proname => 'numeric', prorettype => 'numeric', proargtypes => 'jsonb', prosupport => 'jsonb_cast_support',
prosrc => 'jsonb_numeric' },
-{ oid => '3450', descr => 'convert jsonb to int2',
+{ oid => '3450', descr => 'convert jsonb to int2', prosupport => 'jsonb_cast_support',
proname => 'int2', prorettype => 'int2', proargtypes => 'jsonb',
prosrc => 'jsonb_int2' },
-{ oid => '3451', descr => 'convert jsonb to int4',
+{ oid => '3451', descr => 'convert jsonb to int4', prosupport => 'jsonb_cast_support',
proname => 'int4', prorettype => 'int4', proargtypes => 'jsonb',
prosrc => 'jsonb_int4' },
-{ oid => '3452', descr => 'convert jsonb to int8',
+{ oid => '3452', descr => 'convert jsonb to int8', prosupport => 'jsonb_cast_support',
proname => 'int8', prorettype => 'int8', proargtypes => 'jsonb',
prosrc => 'jsonb_int8' },
-{ oid => '3453', descr => 'convert jsonb to float4',
+{ oid => '3453', descr => 'convert jsonb to float4', prosupport => 'jsonb_cast_support',
proname => 'float4', prorettype => 'float4', proargtypes => 'jsonb',
prosrc => 'jsonb_float4' },
-{ oid => '2580', descr => 'convert jsonb to float8',
+{ oid => '2580', descr => 'convert jsonb to float8', prosupport => 'jsonb_cast_support',
proname => 'float8', prorettype => 'float8', proargtypes => 'jsonb',
prosrc => 'jsonb_float8' },
@@ -9983,6 +9983,30 @@
proname => 'jsonb_object_field_text', prorettype => 'text',
proargtypes => 'jsonb text', proargnames => '{from_json, field_name}',
prosrc => 'jsonb_object_field_text' },
+{ oid => '4552', descr => 'extract jsonbvalue from jsonb for the given field',
+ proname => 'jsonb_object_field_start', prorettype => 'internal',
+ proargtypes => 'internal text', proargnames => '{from_json, field_name}',
+ prosrc => 'jsonb_object_field_start' },
+{ oid => '9303', descr => 'extract josnbvalue from jsonb array for the given index',
+ proname => 'jsonb_array_element_start', prorettype => 'internal',
+ proargtypes => 'internal int4', proargnames => '{from_jsonb, element_index}',
+ prosrc => 'jsonb_array_element_start' },
+{ oid => '4551', descr => 'extract jsonbvalue from jsonb for the given paths',
+ proname => 'jsonb_extract_path_start', provariadic => 'text', prorettype => 'internal',
+ proargtypes => 'internal _text', proallargtypes => '{internal,_text}',
+ proargmodes => '{i,v}', proargnames => '{from_jsonb,path_elems}',
+ prosrc => 'jsonb_extract_path_start'},
+{ oid => '4553', descr => 'convert a jsonbvalue to numeric',
+ proname => 'jsonb_finish_numeric', prorettype => 'numeric',
+ proargtypes => 'internal oid', proargnames => '{from_jsonvalue,target_oid}',
+ prosrc => 'jsonb_finish_numeric' },
+{ oid => '4554', descr => 'convert a jsonbvalue to boolean',
+ proname => 'jsonb_finish_bool', prorettype => 'bool',
+ proargtypes => 'internal', proargnames => '{jsonvalue}',
+ prosrc => 'jsonb_finish_bool' },
+{ oid => '3814', descr => 'planner support for numeric(jsonb)',
+ proname => 'jsonb_cast_support', prorettype => 'internal',
+ proargtypes => 'internal', prosrc => 'jsonb_cast_support' },
{ oid => '3215',
proname => 'jsonb_array_element', prorettype => 'jsonb',
proargtypes => 'jsonb int4', proargnames => '{from_json, element_index}',
@@ -10175,6 +10199,10 @@
proname => 'jsonb_path_query', prorows => '1000', proretset => 't',
prorettype => 'jsonb', proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query' },
+{ oid => '4557', descr => 'jsonpath query as jsonbvalue',
+ proname => 'jsonb_path_query_start', prorows => '1000', proretset => 't',
+ prorettype => 'internal', proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_start' },
{ oid => '4007', descr => 'jsonpath query wrapped into array',
proname => 'jsonb_path_query_array', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
@@ -10183,6 +10211,10 @@
proname => 'jsonb_path_query_first', prorettype => 'jsonb',
proargtypes => 'jsonb jsonpath jsonb bool',
prosrc => 'jsonb_path_query_first' },
+{ oid => '4555', descr => 'jsonpath query first item as jsonbvalue',
+ proname => 'jsonb_path_query_first_start', prorettype => 'internal',
+ proargtypes => 'internal jsonpath jsonb bool',
+ prosrc => 'jsonb_path_query_first_start' },
{ oid => '4009', descr => 'jsonpath match',
proname => 'jsonb_path_match', prorettype => 'bool',
proargtypes => 'jsonb jsonpath jsonb bool', prosrc => 'jsonb_path_match' },
diff --git a/src/include/utils/jsonb.h b/src/include/utils/jsonb.h
index e38dfd49..53e06a20 100644
--- a/src/include/utils/jsonb.h
+++ b/src/include/utils/jsonb.h
@@ -294,6 +294,13 @@ struct JsonbValue
} val;
};
+typedef enum JsonbValueTarget
+{
+ JsonbValue_AsJsonbValue,
+ JsonbValue_AsJsonb,
+ JsonbValue_AsText,
+} JsonbValueTarget;
+
#define IsAJsonbScalar(jsonbval) (((jsonbval)->type >= jbvNull && \
(jsonbval)->type <= jbvBool) || \
(jsonbval)->type == jbvDatetime)
@@ -428,12 +435,12 @@ extern const char *JsonbTypeName(JsonbValue *val);
extern Datum jsonb_set_element(Jsonb *jb, Datum *path, int path_len,
JsonbValue *newval);
extern Datum jsonb_get_element(Jsonb *jb, Datum *path, int npath,
- bool *isnull, bool as_text);
+ bool *isnull, JsonbValueTarget target);
extern bool to_jsonb_is_immutable(Oid typoid);
extern Datum jsonb_build_object_worker(int nargs, const Datum *args, const bool *nulls,
const Oid *types, bool absent_on_null,
bool unique_keys);
extern Datum jsonb_build_array_worker(int nargs, const Datum *args, const bool *nulls,
const Oid *types, bool absent_on_null);
-
+extern Datum jsonbvalue_covert(JsonbValue *jbv, JsonbValueTarget target);
#endif /* __JSONB_H__ */
diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expected/jsonb.out
index b597d01a..0167b0e2 100644
--- a/src/test/regress/expected/jsonb.out
+++ b/src/test/regress/expected/jsonb.out
@@ -457,8 +457,242 @@ CREATE TEMP TABLE test_jsonb (
);
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4
+ ,"field5": [1,2,3], "field6": {"f1":9}
+ ,"field7": true, "field8": [1,2,3,4,5]}');
+\pset null NULL
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+--------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '21'::oid))::smallint
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int4
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int8
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '20'::oid))::bigint
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::float4
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '700'::oid))::real
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::float8
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field4'::text), '701'::oid))::double precision
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json->'field5' -> 0)::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 0), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json->'field5' -> 10)::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_array_element_start(((test_json -> 'field5'::text))::internal, 10), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field6", "f1"}')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f1}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field6", "f2"}')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field6,f2}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_bool(jsonb_extract_path_start((test_json)::internal, VARIADIC '{field7}'::text[]))
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+ numeric | int2 | int4 | int8 | float4 | float8 | numeric | numeric | numeric | numeric | bool
+---------+------+------+------+--------+--------+---------+---------+---------+---------+------
+ 4 | 4 | 4 | 4 | 4 | 4 | 1 | NULL | 9 | NULL | t
+(1 row)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')
+FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------------------------
+ ProjectSet
+ Output: jsonb_path_query(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)
+ -> Seq Scan on pg_temp.test_jsonb
+ Output: json_type, test_json
+ Filter: (test_jsonb.json_type = 'object'::text)
+(5 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')
+FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_path_query_first(test_json, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Result
+ Output: (jsonb_finish_numeric((jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)), '21'::oid))::smallint
+ -> ProjectSet
+ Output: jsonb_path_query_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false)
+ -> Seq Scan on pg_temp.test_jsonb
+ Output: json_type, test_json
+ Filter: (test_jsonb.json_type = 'object'::text)
+(7 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_path_query_first_start((test_json)::internal, '$."field8"[*]?(@ >= $"min" && @ <= $"max")'::jsonpath, '{"max": 4, "min": 2}'::jsonb, false), '21'::oid))::smallint
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+ jsonb_path_query | jsonb_path_query_first | jsonb_path_query | jsonb_path_query_first
+------------------+------------------------+------------------+------------------------
+ 2 | 2 | 2 | 2
+ 3 | 2 | 3 | 2
+ 4 | 2 | 4 | 2
+(3 rows)
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: jsonb_finish_numeric(jsonb_extract_path_start((test_json)::internal, VARIADIC '{}'::text[]), '1700'::oid)
+ Filter: (test_jsonb.json_type = 'scalarint'::text)
+(3 rows)
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+ numeric
+---------
+ 2
+(1 row)
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------------
+ Seq Scan on pg_temp.test_jsonb
+ Output: (jsonb_finish_numeric(jsonb_object_field_start((test_json)::internal, 'field1'::text), '23'::oid))::integer
+ Filter: (test_jsonb.json_type = 'object'::text)
+(3 rows)
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type integer
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+ERROR: cannot cast jsonb string to type boolean
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
?column?
----------
@@ -586,7 +820,9 @@ SELECT jsonb_object_keys(test_json) FROM test_jsonb WHERE json_type = 'object';
field4
field5
field6
-(6 rows)
+ field7
+ field8
+(8 rows)
-- nulls
SELECT (test_json->'field3') IS NULL AS expect_false FROM test_jsonb WHERE json_type = 'object';
diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.sql
index 6dae715a..1b9959f9 100644
--- a/src/test/regress/sql/jsonb.sql
+++ b/src/test/regress/sql/jsonb.sql
@@ -154,8 +154,122 @@ CREATE TEMP TABLE test_jsonb (
INSERT INTO test_jsonb VALUES
('scalar','"a scalar"'),
+('scalarint','2'),
('array','["zero", "one","two",null,"four","five", [1,2,3],{"f1":9}]'),
-('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4, "field5": [1,2,3], "field6": {"f1":9}}');
+('object','{"field1":"val1","field2":"val2","field3":null, "field4": 4
+ ,"field5": [1,2,3], "field6": {"f1":9}
+ ,"field7": true, "field8": [1,2,3,4,5]}');
+
+\pset null NULL
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int4
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::int8
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::float4
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field4')::float8
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json->'field5' -> 0)::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json->'field5' -> 10)::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field6", "f1"}')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field6", "f2"}')::numeric
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+SELECT
+(test_json -> 'field4')::numeric,
+(test_json -> 'field4')::int2,
+(test_json -> 'field4')::int4,
+(test_json -> 'field4')::int8,
+(test_json -> 'field4')::float4,
+(test_json -> 'field4')::float8,
+(test_json -> 'field5' -> 0)::numeric,
+(test_json -> 'field5' -> 10)::numeric,
+(test_json #> '{"field6", "f1"}')::numeric,
+(test_json #> '{"field6", "f2"}')::numeric,
+(test_json#>'{"field7"}')::bool
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')
+FROM test_jsonb WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')
+FROM test_jsonb WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb WHERE json_type = 'object';
+
+SELECT
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}'),
+jsonb_path_query(test_json,'$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2,
+jsonb_path_query_first(test_json, '$.field8[*] ? (@ >= $min && @ <= $max)', '{"min":2, "max":4}')::int2
+FROM test_jsonb
+WHERE json_type = 'object';
+
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+SELECT (test_json #> '{}')::numeric FROM test_jsonb WHERE json_type = 'scalarint';
+
+-- let raise some errors.
+EXPLAIN (COSTS OFF, VERBOSE)
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::int4 FROM test_jsonb WHERE json_type = 'object';
+
+SELECT (test_json -> 'field1')::bool FROM test_jsonb WHERE json_type = 'object';
+
+\pset null ''
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'scalar';
SELECT test_json -> 'x' FROM test_jsonb WHERE json_type = 'array';
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 5fd46b7b..4ca6679e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1265,6 +1265,7 @@ JsonArrayAgg
JsonArrayConstructor
JsonArrayQueryConstructor
JsonBaseObjectInfo
+JsonbValueTarget
JsonConstructorExpr
JsonConstructorExprState
JsonConstructorType
--
2.34.1
[application/octet-stream] v1-0001-Improve-performance-of-Jsonb-extract-via-key-and-c.no-cfbot (20.7K, 3-v1-0001-Improve-performance-of-Jsonb-extract-via-key-and-c.no-cfbot)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
end of thread, other threads:[~2024-01-08 00:00 UTC | newest]
Thread overview: 35+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-08-16 16:32 Re: Extract numeric filed in JSONB more effectively jian he <[email protected]>
2023-08-17 09:07 ` Andy Fan <[email protected]>
2023-08-17 20:30 ` Chapman Flack <[email protected]>
2023-08-18 01:14 ` Andy Fan <[email protected]>
2023-08-18 02:55 ` Chapman Flack <[email protected]>
2023-08-18 05:02 ` jian he <[email protected]>
2023-08-18 07:41 ` Andy Fan <[email protected]>
2023-08-18 18:50 ` Chapman Flack <[email protected]>
2023-08-18 19:08 ` Chapman Flack <[email protected]>
2023-08-18 21:02 ` Chapman Flack <[email protected]>
2023-08-21 10:58 ` Andy Fan <[email protected]>
2023-08-21 01:31 ` Andy Fan <[email protected]>
2023-08-21 03:19 ` Chapman Flack <[email protected]>
2023-08-22 03:14 ` Andy Fan <[email protected]>
2023-08-22 05:54 ` Andy Fan <[email protected]>
2023-08-22 12:16 ` Chapman Flack <[email protected]>
2023-08-26 22:28 ` Chapman Flack <[email protected]>
2023-08-30 04:47 ` Andy Fan <[email protected]>
2023-08-30 13:47 ` Chapman Flack <[email protected]>
2023-08-31 09:10 ` Andy Fan <[email protected]>
2023-09-01 03:09 ` Andy Fan <[email protected]>
2023-09-02 01:25 ` jian he <[email protected]>
2023-09-04 11:43 ` Andy Fan <[email protected]>
2023-09-04 14:35 ` Andy Fan <[email protected]>
2023-09-05 12:51 ` jian he <[email protected]>
2023-09-06 07:00 ` Andy Fan <[email protected]>
2023-09-13 21:18 ` Chapman Flack <[email protected]>
2023-09-14 06:41 ` Andy Fan <[email protected]>
2023-09-15 01:53 ` Andy Fan <[email protected]>
2023-10-05 06:53 ` Andy Fan <[email protected]>
2023-11-01 02:17 ` Chapman Flack <[email protected]>
2023-11-06 03:26 ` [email protected]
2024-01-02 08:18 ` jian he <[email protected]>
2024-01-07 07:17 ` Andy Fan <[email protected]>
2024-01-08 00:00 ` jian he <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox