Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p45gX-0004qi-Rv for pgsql-hackers@arkaria.postgresql.org; Sat, 10 Dec 2022 19:38:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1p45gW-0007lE-AH for pgsql-hackers@arkaria.postgresql.org; Sat, 10 Dec 2022 19:38:56 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p45gV-0007l5-GH for pgsql-hackers@lists.postgresql.org; Sat, 10 Dec 2022 19:38:55 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p45gM-0000q1-7t for pgsql-hackers@lists.postgresql.org; Sat, 10 Dec 2022 19:38:54 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 2BAJcccM1436687; Sat, 10 Dec 2022 14:38:38 -0500 From: Tom Lane To: Andrew Dunstan cc: Andres Freund , Alexander Korotkov , Robert Haas , "David G. Johnston" , Corey Huinker , Alvaro Herrera , Joe Conway , Nikita Glukhov , PostgreSQL Hackers Subject: Re: Error-safe user functions In-reply-to: <3bac9841-fe07-713d-fa42-606c225567d6@dunslane.net> References: <4009543.1670422833@sss.pgh.pa.us> <6acddf68-7cb7-e9e7-a5a7-0abc87b3c55b@dunslane.net> <4111458.1670444176@sss.pgh.pa.us> <3211.1670452341@sss.pgh.pa.us> <20221207233518.sudtaxn6omtvpzvr@awork3.anarazel.de> <186288.1670517119@sss.pgh.pa.us> <20221208215836.3naznqmal2m32hzc@awork3.anarazel.de> <512551.1670540229@sss.pgh.pa.us> <20221209023351.l362yejerfoq3lfp@awork3.anarazel.de> <633426.1670554790@sss.pgh.pa.us> <01ff2051-806d-af2f-4f9e-793f7ba9a3f8@dunslane.net> <778369.1670598993@sss.pgh.pa.us> <5aaf7c66-9df9-21a9-1ff4-02c7848def52@dunslane.net> <3bac9841-fe07-713d-fa42-606c225567d6@dunslane.net> Comments: In-reply-to Andrew Dunstan message dated "Sat, 10 Dec 2022 09:35:12 -0500" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0" Content-ID: <1436571.1670700998.0@sss.pgh.pa.us> Date: Sat, 10 Dec 2022 14:38:38 -0500 Message-ID: <1436686.1670701118@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <1436571.1670700998.1@sss.pgh.pa.us> Andrew Dunstan writes: > OK, json is a fairly easy case, see attached. But jsonb is a different > kettle of fish. Both the semantic routines called by the parser and the > subsequent call to JsonbValueToJsonb() can raise errors. These are > pretty much all about breaking various limits (for strings, objects, > arrays). There's also a call to numeric_in, but I assume that a string > that's already parsed as a valid json numeric literal won't upset > numeric_in. Um, nope ... regression=# select '1e1000000'::jsonb; ERROR: value overflows numeric format LINE 1: select '1e1000000'::jsonb; ^ > Many of these occur several calls down the stack, so > adjusting everything to deal with them would be fairly invasive. Perhaps > we could instead document that this class of input error won't be > trapped, at least for jsonb. Seeing that SQL/JSON is one of the major drivers of this whole project, it seemed a little sad to me that jsonb couldn't manage to implement what is required. So I spent a bit of time poking at it. Attached is an extended version of your patch that also covers jsonb. The main thing I soon realized is that the JsonSemAction API is based on the assumption that semantic actions will report errors by throwing them. This is a bit schizophrenic considering the parser itself carefully hands back error codes instead of throwing anything (excluding palloc failures of course). What I propose in the attached is that we change that API so that action functions return JsonParseErrorType, and add an enum value denoting "I already logged a suitable error, so you don't have to". It was a little tedious to modify all the existing functions that way, but not hard. Only the ones used by jsonb_in need to do anything except "return JSON_SUCCESS", at least for now. (I wonder if pg_verifybackup's parse_manifest.c could use a second look at how it's handling errors, given this API. I didn't study it closely.) I have not done anything here about errors within JsonbValueToJsonb. There would need to be another round of API-extension in that area if we want to be able to trap its errors. As you say, those are mostly about exceeding implementation size limits, so I suppose one could argue that they are not so different from palloc failure. It's still annoying. If people are good with the changes attached, I might take a look at that. regards, tom lane ------- =_aaaaaaaaaa0 Content-Type: text/x-diff; name="0001-fix-json-and-jsonb.patch"; charset="us-ascii" Content-ID: <1436571.1670700998.2@sss.pgh.pa.us> Content-Description: 0001-fix-json-and-jsonb.patch Content-Transfer-Encoding: quoted-printable diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c index fee2ffb55c..e6896eccfe 100644 --- a/src/backend/utils/adt/json.c +++ b/src/backend/utils/adt/json.c @@ -81,9 +81,10 @@ json_in(PG_FUNCTION_ARGS) = /* validate it */ lex =3D makeJsonLexContext(result, false); - pg_parse_json_or_ereport(lex, &nullSemAction); + if (!pg_parse_json_or_errsave(lex, &nullSemAction, fcinfo->context)) + PG_RETURN_NULL(); = - /* Internal representation is the same as text, for now */ + /* Internal representation is the same as text */ PG_RETURN_TEXT_P(result); } = @@ -1337,7 +1338,7 @@ json_typeof(PG_FUNCTION_ARGS) /* Lex exactly one token from the input and check its type. */ result =3D json_lex(lex); if (result !=3D JSON_SUCCESS) - json_ereport_error(result, lex); + json_errsave_error(result, lex, NULL); tok =3D lex->token_type; switch (tok) { diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c index 9e14922ec2..7c1e5e6144 100644 --- a/src/backend/utils/adt/jsonb.c +++ b/src/backend/utils/adt/jsonb.c @@ -33,6 +33,7 @@ typedef struct JsonbInState { JsonbParseState *parseState; JsonbValue *res; + Node *escontext; } JsonbInState; = /* unlike with json categories, we need to treat json and jsonb different= ly */ @@ -61,15 +62,15 @@ typedef struct JsonbAggState Oid val_output_func; } JsonbAggState; = -static inline Datum jsonb_from_cstring(char *json, int len); -static size_t checkStringLen(size_t len); -static void jsonb_in_object_start(void *pstate); -static void jsonb_in_object_end(void *pstate); -static void jsonb_in_array_start(void *pstate); -static void jsonb_in_array_end(void *pstate); -static void jsonb_in_object_field_start(void *pstate, char *fname, bool i= snull); +static inline Datum jsonb_from_cstring(char *json, int len, Node *esconte= xt); +static bool checkStringLen(size_t len, Node *escontext); +static JsonParseErrorType jsonb_in_object_start(void *pstate); +static JsonParseErrorType jsonb_in_object_end(void *pstate); +static JsonParseErrorType jsonb_in_array_start(void *pstate); +static JsonParseErrorType jsonb_in_array_end(void *pstate); +static JsonParseErrorType jsonb_in_object_field_start(void *pstate, char = *fname, bool isnull); static void jsonb_put_escaped_value(StringInfo out, JsonbValue *scalarVal= ); -static void jsonb_in_scalar(void *pstate, char *token, JsonTokenType toke= ntype); +static JsonParseErrorType jsonb_in_scalar(void *pstate, char *token, Json= TokenType tokentype); static void jsonb_categorize_type(Oid typoid, JsonbTypeCategory *tcategory, Oid *outfuncoid); @@ -98,7 +99,7 @@ jsonb_in(PG_FUNCTION_ARGS) { char *json =3D PG_GETARG_CSTRING(0); = - return jsonb_from_cstring(json, strlen(json)); + return jsonb_from_cstring(json, strlen(json), fcinfo->context); } = /* @@ -122,7 +123,7 @@ jsonb_recv(PG_FUNCTION_ARGS) else elog(ERROR, "unsupported jsonb version number %d", version); = - return jsonb_from_cstring(str, nbytes); + return jsonb_from_cstring(str, nbytes, NULL); } = /* @@ -251,9 +252,12 @@ jsonb_typeof(PG_FUNCTION_ARGS) * Turns json string into a jsonb Datum. * * Uses the json parser (with hooks) to construct a jsonb. + * + * If escontext points to an ErrorSaveContext, errors are reported there + * instead of being thrown. */ static inline Datum -jsonb_from_cstring(char *json, int len) +jsonb_from_cstring(char *json, int len, Node *escontext) { JsonLexContext *lex; JsonbInState state; @@ -263,6 +267,7 @@ jsonb_from_cstring(char *json, int len) memset(&sem, 0, sizeof(sem)); lex =3D makeJsonLexContextCstringLen(json, len, GetDatabaseEncoding(), t= rue); = + state.escontext =3D escontext; sem.semstate =3D (void *) &state; = sem.object_start =3D jsonb_in_object_start; @@ -272,58 +277,67 @@ jsonb_from_cstring(char *json, int len) sem.scalar =3D jsonb_in_scalar; sem.object_field_start =3D jsonb_in_object_field_start; = - pg_parse_json_or_ereport(lex, &sem); + if (!pg_parse_json_or_errsave(lex, &sem, escontext)) + return (Datum) 0; = /* after parsing, the item member has the composed jsonb structure */ PG_RETURN_POINTER(JsonbValueToJsonb(state.res)); } = -static size_t -checkStringLen(size_t len) +static bool +checkStringLen(size_t len, Node *escontext) { if (len > JENTRY_OFFLENMASK) - ereport(ERROR, + ereturn(escontext, false, (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), errmsg("string too long to represent as jsonb string"), errdetail("Due to an implementation restriction, jsonb strings canno= t exceed %d bytes.", JENTRY_OFFLENMASK))); = - return len; + return true; } = -static void +static JsonParseErrorType jsonb_in_object_start(void *pstate) { JsonbInState *_state =3D (JsonbInState *) pstate; = _state->res =3D pushJsonbValue(&_state->parseState, WJB_BEGIN_OBJECT, NU= LL); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType jsonb_in_object_end(void *pstate) { JsonbInState *_state =3D (JsonbInState *) pstate; = _state->res =3D pushJsonbValue(&_state->parseState, WJB_END_OBJECT, NULL= ); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType jsonb_in_array_start(void *pstate) { JsonbInState *_state =3D (JsonbInState *) pstate; = _state->res =3D pushJsonbValue(&_state->parseState, WJB_BEGIN_ARRAY, NUL= L); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType jsonb_in_array_end(void *pstate) { JsonbInState *_state =3D (JsonbInState *) pstate; = _state->res =3D pushJsonbValue(&_state->parseState, WJB_END_ARRAY, NULL)= ; + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType jsonb_in_object_field_start(void *pstate, char *fname, bool isnull) { JsonbInState *_state =3D (JsonbInState *) pstate; @@ -331,10 +345,14 @@ jsonb_in_object_field_start(void *pstate, char *fnam= e, bool isnull) = Assert(fname !=3D NULL); v.type =3D jbvString; - v.val.string.len =3D checkStringLen(strlen(fname)); + v.val.string.len =3D strlen(fname); + if (!checkStringLen(v.val.string.len, _state->escontext)) + return JSON_SEM_ACTION_FAILED; v.val.string.val =3D fname; = _state->res =3D pushJsonbValue(&_state->parseState, WJB_KEY, &v); + + return JSON_SUCCESS; } = static void @@ -367,7 +385,7 @@ jsonb_put_escaped_value(StringInfo out, JsonbValue *sc= alarVal) /* * For jsonb we always want the de-escaped value - that's what's in token */ -static void +static JsonParseErrorType jsonb_in_scalar(void *pstate, char *token, JsonTokenType tokentype) { JsonbInState *_state =3D (JsonbInState *) pstate; @@ -380,7 +398,9 @@ jsonb_in_scalar(void *pstate, char *token, JsonTokenTy= pe tokentype) case JSON_TOKEN_STRING: Assert(token !=3D NULL); v.type =3D jbvString; - v.val.string.len =3D checkStringLen(strlen(token)); + v.val.string.len =3D strlen(token); + if (!checkStringLen(v.val.string.len, _state->escontext)) + return JSON_SEM_ACTION_FAILED; v.val.string.val =3D token; break; case JSON_TOKEN_NUMBER: @@ -391,10 +411,11 @@ jsonb_in_scalar(void *pstate, char *token, JsonToken= Type tokentype) */ Assert(token !=3D NULL); v.type =3D jbvNumeric; - numd =3D DirectFunctionCall3(numeric_in, - CStringGetDatum(token), - ObjectIdGetDatum(InvalidOid), - Int32GetDatum(-1)); + if (!DirectInputFunctionCallSafe(numeric_in, token, + InvalidOid, -1, + _state->escontext, + &numd)) + return JSON_SEM_ACTION_FAILED; v.val.numeric =3D DatumGetNumeric(numd); break; case JSON_TOKEN_TRUE: @@ -443,6 +464,8 @@ jsonb_in_scalar(void *pstate, char *token, JsonTokenTy= pe tokentype) elog(ERROR, "unexpected parent of nested structure"); } } + + return JSON_SUCCESS; } = /* @@ -726,6 +749,9 @@ jsonb_categorize_type(Oid typoid, * * If key_scalar is true, the value is stored as a key, so insist * it's of an acceptable type, and force it to be a jbvString. + * + * Note: currently, we assume that result->escontext is NULL and errors + * will be thrown. */ static void datum_to_jsonb(Datum val, bool is_null, JsonbInState *result, @@ -898,7 +924,8 @@ datum_to_jsonb(Datum val, bool is_null, JsonbInState *= result, default: outputstr =3D OidOutputFunctionCall(outfuncoid, val); jb.type =3D jbvString; - jb.val.string.len =3D checkStringLen(strlen(outputstr)); + jb.val.string.len =3D strlen(outputstr); + (void) checkStringLen(jb.val.string.len, NULL); jb.val.string.val =3D outputstr; break; } @@ -1636,6 +1663,7 @@ jsonb_agg_finalfn(PG_FUNCTION_ARGS) * shallow clone is sufficient as we aren't going to change any of the * values, just add the final array end marker. */ + memset(&result, 0, sizeof(JsonbInState)); = result.parseState =3D clone_parse_state(arg->res->parseState); = @@ -1868,6 +1896,7 @@ jsonb_object_agg_finalfn(PG_FUNCTION_ARGS) * going to change any of the values, just add the final object end * marker. */ + memset(&result, 0, sizeof(JsonbInState)); = result.parseState =3D clone_parse_state(arg->res->parseState); = diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jso= nfuncs.c index bfc3f02a86..463a8fdf23 100644 --- a/src/backend/utils/adt/jsonfuncs.c +++ b/src/backend/utils/adt/jsonfuncs.c @@ -25,6 +25,7 @@ #include "lib/stringinfo.h" #include "mb/pg_wchar.h" #include "miscadmin.h" +#include "nodes/miscnodes.h" #include "utils/array.h" #include "utils/builtins.h" #include "utils/fmgroids.h" @@ -336,20 +337,20 @@ typedef struct JsObject static int report_json_context(JsonLexContext *lex); = /* semantic action functions for json_object_keys */ -static void okeys_object_field_start(void *state, char *fname, bool isnul= l); -static void okeys_array_start(void *state); -static void okeys_scalar(void *state, char *token, JsonTokenType tokentyp= e); +static JsonParseErrorType okeys_object_field_start(void *state, char *fna= me, bool isnull); +static JsonParseErrorType okeys_array_start(void *state); +static JsonParseErrorType okeys_scalar(void *state, char *token, JsonToke= nType tokentype); = /* semantic action functions for json_get* functions */ -static void get_object_start(void *state); -static void get_object_end(void *state); -static void get_object_field_start(void *state, char *fname, bool isnull)= ; -static void get_object_field_end(void *state, char *fname, bool isnull); -static void get_array_start(void *state); -static void get_array_end(void *state); -static void get_array_element_start(void *state, bool isnull); -static void get_array_element_end(void *state, bool isnull); -static void get_scalar(void *state, char *token, JsonTokenType tokentype)= ; +static JsonParseErrorType get_object_start(void *state); +static JsonParseErrorType get_object_end(void *state); +static JsonParseErrorType get_object_field_start(void *state, char *fname= , bool isnull); +static JsonParseErrorType get_object_field_end(void *state, char *fname, = bool isnull); +static JsonParseErrorType get_array_start(void *state); +static JsonParseErrorType get_array_end(void *state); +static JsonParseErrorType get_array_element_start(void *state, bool isnul= l); +static JsonParseErrorType get_array_element_end(void *state, bool isnull)= ; +static JsonParseErrorType get_scalar(void *state, char *token, JsonTokenT= ype tokentype); = /* common worker function for json getter functions */ static Datum get_path_all(FunctionCallInfo fcinfo, bool as_text); @@ -359,9 +360,9 @@ static Datum get_jsonb_path_all(FunctionCallInfo fcinf= o, bool as_text); static text *JsonbValueAsText(JsonbValue *v); = /* semantic action functions for json_array_length */ -static void alen_object_start(void *state); -static void alen_scalar(void *state, char *token, JsonTokenType tokentype= ); -static void alen_array_element_start(void *state, bool isnull); +static JsonParseErrorType alen_object_start(void *state); +static JsonParseErrorType alen_scalar(void *state, char *token, JsonToken= Type tokentype); +static JsonParseErrorType alen_array_element_start(void *state, bool isnu= ll); = /* common workers for json{b}_each* functions */ static Datum each_worker(FunctionCallInfo fcinfo, bool as_text); @@ -369,10 +370,10 @@ static Datum each_worker_jsonb(FunctionCallInfo fcin= fo, const char *funcname, bool as_text); = /* semantic action functions for json_each */ -static void each_object_field_start(void *state, char *fname, bool isnull= ); -static void each_object_field_end(void *state, char *fname, bool isnull); -static void each_array_start(void *state); -static void each_scalar(void *state, char *token, JsonTokenType tokentype= ); +static JsonParseErrorType each_object_field_start(void *state, char *fnam= e, bool isnull); +static JsonParseErrorType each_object_field_end(void *state, char *fname,= bool isnull); +static JsonParseErrorType each_array_start(void *state); +static JsonParseErrorType each_scalar(void *state, char *token, JsonToken= Type tokentype); = /* common workers for json{b}_array_elements_* functions */ static Datum elements_worker(FunctionCallInfo fcinfo, const char *funcnam= e, @@ -381,44 +382,44 @@ static Datum elements_worker_jsonb(FunctionCallInfo = fcinfo, const char *funcname bool as_text); = /* semantic action functions for json_array_elements */ -static void elements_object_start(void *state); -static void elements_array_element_start(void *state, bool isnull); -static void elements_array_element_end(void *state, bool isnull); -static void elements_scalar(void *state, char *token, JsonTokenType token= type); +static JsonParseErrorType elements_object_start(void *state); +static JsonParseErrorType elements_array_element_start(void *state, bool = isnull); +static JsonParseErrorType elements_array_element_end(void *state, bool is= null); +static JsonParseErrorType elements_scalar(void *state, char *token, JsonT= okenType tokentype); = /* turn a json object into a hash table */ static HTAB *get_json_object_as_hash(char *json, int len, const char *fun= cname); = /* semantic actions for populate_array_json */ -static void populate_array_object_start(void *_state); -static void populate_array_array_end(void *_state); -static void populate_array_element_start(void *_state, bool isnull); -static void populate_array_element_end(void *_state, bool isnull); -static void populate_array_scalar(void *_state, char *token, JsonTokenTyp= e tokentype); +static JsonParseErrorType populate_array_object_start(void *_state); +static JsonParseErrorType populate_array_array_end(void *_state); +static JsonParseErrorType populate_array_element_start(void *_state, bool= isnull); +static JsonParseErrorType populate_array_element_end(void *_state, bool i= snull); +static JsonParseErrorType populate_array_scalar(void *_state, char *token= , JsonTokenType tokentype); = /* semantic action functions for get_json_object_as_hash */ -static void hash_object_field_start(void *state, char *fname, bool isnull= ); -static void hash_object_field_end(void *state, char *fname, bool isnull); -static void hash_array_start(void *state); -static void hash_scalar(void *state, char *token, JsonTokenType tokentype= ); +static JsonParseErrorType hash_object_field_start(void *state, char *fnam= e, bool isnull); +static JsonParseErrorType hash_object_field_end(void *state, char *fname,= bool isnull); +static JsonParseErrorType hash_array_start(void *state); +static JsonParseErrorType hash_scalar(void *state, char *token, JsonToken= Type tokentype); = /* semantic action functions for populate_recordset */ -static void populate_recordset_object_field_start(void *state, char *fnam= e, bool isnull); -static void populate_recordset_object_field_end(void *state, char *fname,= bool isnull); -static void populate_recordset_scalar(void *state, char *token, JsonToken= Type tokentype); -static void populate_recordset_object_start(void *state); -static void populate_recordset_object_end(void *state); -static void populate_recordset_array_start(void *state); -static void populate_recordset_array_element_start(void *state, bool isnu= ll); +static JsonParseErrorType populate_recordset_object_field_start(void *sta= te, char *fname, bool isnull); +static JsonParseErrorType populate_recordset_object_field_end(void *state= , char *fname, bool isnull); +static JsonParseErrorType populate_recordset_scalar(void *state, char *to= ken, JsonTokenType tokentype); +static JsonParseErrorType populate_recordset_object_start(void *state); +static JsonParseErrorType populate_recordset_object_end(void *state); +static JsonParseErrorType populate_recordset_array_start(void *state); +static JsonParseErrorType populate_recordset_array_element_start(void *st= ate, bool isnull); = /* semantic action functions for json_strip_nulls */ -static void sn_object_start(void *state); -static void sn_object_end(void *state); -static void sn_array_start(void *state); -static void sn_array_end(void *state); -static void sn_object_field_start(void *state, char *fname, bool isnull); -static void sn_array_element_start(void *state, bool isnull); -static void sn_scalar(void *state, char *token, JsonTokenType tokentype); +static JsonParseErrorType sn_object_start(void *state); +static JsonParseErrorType sn_object_end(void *state); +static JsonParseErrorType sn_array_start(void *state); +static JsonParseErrorType sn_array_end(void *state); +static JsonParseErrorType sn_object_field_start(void *state, char *fname,= bool isnull); +static JsonParseErrorType sn_array_element_start(void *state, bool isnull= ); +static JsonParseErrorType sn_scalar(void *state, char *token, JsonTokenTy= pe tokentype); = /* worker functions for populate_record, to_record, populate_recordset an= d to_recordset */ static Datum populate_recordset_worker(FunctionCallInfo fcinfo, const cha= r *funcname, @@ -478,33 +479,43 @@ static void setPathArray(JsonbIterator **it, Datum *= path_elems, JsonbValue *newval, uint32 nelems, int op_type); = /* function supporting iterate_json_values */ -static void iterate_values_scalar(void *state, char *token, JsonTokenType= tokentype); -static void iterate_values_object_field_start(void *state, char *fname, b= ool isnull); +static JsonParseErrorType iterate_values_scalar(void *state, char *token,= JsonTokenType tokentype); +static JsonParseErrorType iterate_values_object_field_start(void *state, = char *fname, bool isnull); = /* functions supporting transform_json_string_values */ -static void transform_string_values_object_start(void *state); -static void transform_string_values_object_end(void *state); -static void transform_string_values_array_start(void *state); -static void transform_string_values_array_end(void *state); -static void transform_string_values_object_field_start(void *state, char = *fname, bool isnull); -static void transform_string_values_array_element_start(void *state, bool= isnull); -static void transform_string_values_scalar(void *state, char *token, Json= TokenType tokentype); +static JsonParseErrorType transform_string_values_object_start(void *stat= e); +static JsonParseErrorType transform_string_values_object_end(void *state)= ; +static JsonParseErrorType transform_string_values_array_start(void *state= ); +static JsonParseErrorType transform_string_values_array_end(void *state); +static JsonParseErrorType transform_string_values_object_field_start(void= *state, char *fname, bool isnull); +static JsonParseErrorType transform_string_values_array_element_start(voi= d *state, bool isnull); +static JsonParseErrorType transform_string_values_scalar(void *state, cha= r *token, JsonTokenType tokentype); + = /* - * pg_parse_json_or_ereport + * pg_parse_json_or_errsave * * This function is like pg_parse_json, except that it does not return a * JsonParseErrorType. Instead, in case of any failure, this function wil= l + * save error data into *escontext if that's an ErrorSaveContext, otherwi= se * ereport(ERROR). + * + * Returns a boolean indicating success or failure (failure will only be + * returned when escontext is an ErrorSaveContext). */ -void -pg_parse_json_or_ereport(JsonLexContext *lex, JsonSemAction *sem) +bool +pg_parse_json_or_errsave(JsonLexContext *lex, JsonSemAction *sem, + Node *escontext) { JsonParseErrorType result; = result =3D pg_parse_json(lex, sem); if (result !=3D JSON_SUCCESS) - json_ereport_error(result, lex); + { + json_errsave_error(result, lex, escontext); + return false; + } + return true; } = /* @@ -608,17 +619,24 @@ jsonb_object_keys(PG_FUNCTION_ARGS) * Report a JSON error. */ void -json_ereport_error(JsonParseErrorType error, JsonLexContext *lex) +json_errsave_error(JsonParseErrorType error, JsonLexContext *lex, + Node *escontext) { if (error =3D=3D JSON_UNICODE_HIGH_ESCAPE || error =3D=3D JSON_UNICODE_CODE_POINT_ZERO) - ereport(ERROR, + errsave(escontext, (errcode(ERRCODE_UNTRANSLATABLE_CHARACTER), errmsg("unsupported Unicode escape sequence"), errdetail_internal("%s", json_errdetail(error, lex)), report_json_context(lex))); + else if (error =3D=3D JSON_SEM_ACTION_FAILED) + { + /* semantic action function had better have reported something */ + if (!SOFT_ERROR_OCCURRED(escontext)) + elog(ERROR, "JSON semantic action function did not provide error infor= mation"); + } else - ereport(ERROR, + errsave(escontext, (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), errmsg("invalid input syntax for type %s", "json"), errdetail_internal("%s", json_errdetail(error, lex)), @@ -745,14 +763,14 @@ json_object_keys(PG_FUNCTION_ARGS) SRF_RETURN_DONE(funcctx); } = -static void +static JsonParseErrorType okeys_object_field_start(void *state, char *fname, bool isnull) { OkeysState *_state =3D (OkeysState *) state; = /* only collecting keys for the top level object */ if (_state->lex->lex_level !=3D 1) - return; + return JSON_SUCCESS; = /* enlarge result array if necessary */ if (_state->result_count >=3D _state->result_size) @@ -764,9 +782,11 @@ okeys_object_field_start(void *state, char *fname, bo= ol isnull) = /* save a copy of the field name */ _state->result[_state->result_count++] =3D pstrdup(fname); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType okeys_array_start(void *state) { OkeysState *_state =3D (OkeysState *) state; @@ -777,9 +797,11 @@ okeys_array_start(void *state) (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot call %s on an array", "json_object_keys"))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType okeys_scalar(void *state, char *token, JsonTokenType tokentype) { OkeysState *_state =3D (OkeysState *) state; @@ -790,6 +812,8 @@ okeys_scalar(void *state, char *token, JsonTokenType t= okentype) (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot call %s on a scalar", "json_object_keys"))); + + return JSON_SUCCESS; } = /* @@ -1112,7 +1136,7 @@ get_worker(text *json, return state->tresult; } = -static void +static JsonParseErrorType get_object_start(void *state) { GetState *_state =3D (GetState *) state; @@ -1127,9 +1151,11 @@ get_object_start(void *state) */ _state->result_start =3D _state->lex->token_start; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_object_end(void *state) { GetState *_state =3D (GetState *) state; @@ -1143,9 +1169,11 @@ get_object_end(void *state) = _state->tresult =3D cstring_to_text_with_len(start, len); } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_object_field_start(void *state, char *fname, bool isnull) { GetState *_state =3D (GetState *) state; @@ -1188,9 +1216,11 @@ get_object_field_start(void *state, char *fname, bo= ol isnull) _state->result_start =3D _state->lex->token_start; } } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_object_field_end(void *state, char *fname, bool isnull) { GetState *_state =3D (GetState *) state; @@ -1237,9 +1267,11 @@ get_object_field_end(void *state, char *fname, bool= isnull) /* this should be unnecessary but let's do it for cleanliness: */ _state->result_start =3D NULL; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_array_start(void *state) { GetState *_state =3D (GetState *) state; @@ -1260,7 +1292,7 @@ get_array_start(void *state) = error =3D json_count_array_elements(_state->lex, &nelements); if (error !=3D JSON_SUCCESS) - json_ereport_error(error, _state->lex); + json_errsave_error(error, _state->lex, NULL); = if (-_state->path_indexes[lex_level] <=3D nelements) _state->path_indexes[lex_level] +=3D nelements; @@ -1275,9 +1307,11 @@ get_array_start(void *state) */ _state->result_start =3D _state->lex->token_start; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_array_end(void *state) { GetState *_state =3D (GetState *) state; @@ -1291,9 +1325,11 @@ get_array_end(void *state) = _state->tresult =3D cstring_to_text_with_len(start, len); } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_array_element_start(void *state, bool isnull) { GetState *_state =3D (GetState *) state; @@ -1337,9 +1373,11 @@ get_array_element_start(void *state, bool isnull) _state->result_start =3D _state->lex->token_start; } } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_array_element_end(void *state, bool isnull) { GetState *_state =3D (GetState *) state; @@ -1379,9 +1417,11 @@ get_array_element_end(void *state, bool isnull) = _state->result_start =3D NULL; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType get_scalar(void *state, char *token, JsonTokenType tokentype) { GetState *_state =3D (GetState *) state; @@ -1420,6 +1460,8 @@ get_scalar(void *state, char *token, JsonTokenType t= okentype) /* make sure the next call to get_scalar doesn't overwrite it */ _state->next_scalar =3D false; } + + return JSON_SUCCESS; } = Datum @@ -1834,7 +1876,7 @@ jsonb_array_length(PG_FUNCTION_ARGS) * a scalar or an object). */ = -static void +static JsonParseErrorType alen_object_start(void *state) { AlenState *_state =3D (AlenState *) state; @@ -1844,9 +1886,11 @@ alen_object_start(void *state) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot get array length of a non-array"))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType alen_scalar(void *state, char *token, JsonTokenType tokentype) { AlenState *_state =3D (AlenState *) state; @@ -1856,9 +1900,11 @@ alen_scalar(void *state, char *token, JsonTokenType= tokentype) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot get array length of a scalar"))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType alen_array_element_start(void *state, bool isnull) { AlenState *_state =3D (AlenState *) state; @@ -1866,6 +1912,8 @@ alen_array_element_start(void *state, bool isnull) /* just count up all the level 1 elements */ if (_state->lex->lex_level =3D=3D 1) _state->count++; + + return JSON_SUCCESS; } = /* @@ -2026,7 +2074,7 @@ each_worker(FunctionCallInfo fcinfo, bool as_text) } = = -static void +static JsonParseErrorType each_object_field_start(void *state, char *fname, bool isnull) { EachState *_state =3D (EachState *) state; @@ -2044,9 +2092,11 @@ each_object_field_start(void *state, char *fname, b= ool isnull) else _state->result_start =3D _state->lex->token_start; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType each_object_field_end(void *state, char *fname, bool isnull) { EachState *_state =3D (EachState *) state; @@ -2059,7 +2109,7 @@ each_object_field_end(void *state, char *fname, bool= isnull) = /* skip over nested objects */ if (_state->lex->lex_level !=3D 1) - return; + return JSON_SUCCESS; = /* use the tmp context so we can clean up after each tuple is done */ old_cxt =3D MemoryContextSwitchTo(_state->tmp_cxt); @@ -2090,9 +2140,11 @@ each_object_field_end(void *state, char *fname, boo= l isnull) /* clean up and switch back */ MemoryContextSwitchTo(old_cxt); MemoryContextReset(_state->tmp_cxt); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType each_array_start(void *state) { EachState *_state =3D (EachState *) state; @@ -2102,9 +2154,11 @@ each_array_start(void *state) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot deconstruct an array as an object"))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType each_scalar(void *state, char *token, JsonTokenType tokentype) { EachState *_state =3D (EachState *) state; @@ -2118,6 +2172,8 @@ each_scalar(void *state, char *token, JsonTokenType = tokentype) /* supply de-escaped value if required */ if (_state->next_scalar) _state->normalized_scalar =3D token; + + return JSON_SUCCESS; } = /* @@ -2268,7 +2324,7 @@ elements_worker(FunctionCallInfo fcinfo, const char = *funcname, bool as_text) PG_RETURN_NULL(); } = -static void +static JsonParseErrorType elements_array_element_start(void *state, bool isnull) { ElementsState *_state =3D (ElementsState *) state; @@ -2286,9 +2342,11 @@ elements_array_element_start(void *state, bool isnu= ll) else _state->result_start =3D _state->lex->token_start; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType elements_array_element_end(void *state, bool isnull) { ElementsState *_state =3D (ElementsState *) state; @@ -2301,7 +2359,7 @@ elements_array_element_end(void *state, bool isnull) = /* skip over nested objects */ if (_state->lex->lex_level !=3D 1) - return; + return JSON_SUCCESS; = /* use the tmp context so we can clean up after each tuple is done */ old_cxt =3D MemoryContextSwitchTo(_state->tmp_cxt); @@ -2330,9 +2388,11 @@ elements_array_element_end(void *state, bool isnull= ) /* clean up and switch back */ MemoryContextSwitchTo(old_cxt); MemoryContextReset(_state->tmp_cxt); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType elements_object_start(void *state) { ElementsState *_state =3D (ElementsState *) state; @@ -2343,9 +2403,11 @@ elements_object_start(void *state) (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot call %s on a non-array", _state->function_name))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType elements_scalar(void *state, char *token, JsonTokenType tokentype) { ElementsState *_state =3D (ElementsState *) state; @@ -2360,6 +2422,8 @@ elements_scalar(void *state, char *token, JsonTokenT= ype tokentype) /* supply de-escaped value if required */ if (_state->next_scalar) _state->normalized_scalar =3D token; + + return JSON_SUCCESS; } = /* @@ -2508,7 +2572,7 @@ populate_array_element(PopulateArrayContext *ctx, in= t ndim, JsValue *jsv) } = /* json object start handler for populate_array_json() */ -static void +static JsonParseErrorType populate_array_object_start(void *_state) { PopulateArrayState *state =3D (PopulateArrayState *) _state; @@ -2518,10 +2582,12 @@ populate_array_object_start(void *_state) populate_array_assign_ndims(state->ctx, ndim); else if (ndim < state->ctx->ndims) populate_array_report_expected_array(state->ctx, ndim); + + return JSON_SUCCESS; } = /* json array end handler for populate_array_json() */ -static void +static JsonParseErrorType populate_array_array_end(void *_state) { PopulateArrayState *state =3D (PopulateArrayState *) _state; @@ -2533,10 +2599,12 @@ populate_array_array_end(void *_state) = if (ndim < ctx->ndims) populate_array_check_dimension(ctx, ndim); + + return JSON_SUCCESS; } = /* json array element start handler for populate_array_json() */ -static void +static JsonParseErrorType populate_array_element_start(void *_state, bool isnull) { PopulateArrayState *state =3D (PopulateArrayState *) _state; @@ -2549,10 +2617,12 @@ populate_array_element_start(void *_state, bool is= null) state->element_type =3D state->lex->token_type; state->element_scalar =3D NULL; } + + return JSON_SUCCESS; } = /* json array element end handler for populate_array_json() */ -static void +static JsonParseErrorType populate_array_element_end(void *_state, bool isnull) { PopulateArrayState *state =3D (PopulateArrayState *) _state; @@ -2588,10 +2658,12 @@ populate_array_element_end(void *_state, bool isnu= ll) = populate_array_element(ctx, ndim, &jsv); } + + return JSON_SUCCESS; } = /* json scalar handler for populate_array_json() */ -static void +static JsonParseErrorType populate_array_scalar(void *_state, char *token, JsonTokenType tokentype) { PopulateArrayState *state =3D (PopulateArrayState *) _state; @@ -2610,6 +2682,8 @@ populate_array_scalar(void *_state, char *token, Jso= nTokenType tokentype) /* element_type must already be set in populate_array_element_start() *= / Assert(state->element_type =3D=3D tokentype); } + + return JSON_SUCCESS; } = /* parse a json array and populate array */ @@ -3491,13 +3565,13 @@ get_json_object_as_hash(char *json, int len, const= char *funcname) return tab; } = -static void +static JsonParseErrorType hash_object_field_start(void *state, char *fname, bool isnull) { JHashState *_state =3D (JHashState *) state; = if (_state->lex->lex_level > 1) - return; + return JSON_SUCCESS; = /* remember token type */ _state->saved_token_type =3D _state->lex->token_type; @@ -3513,9 +3587,11 @@ hash_object_field_start(void *state, char *fname, b= ool isnull) /* must be a scalar */ _state->save_json_start =3D NULL; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType hash_object_field_end(void *state, char *fname, bool isnull) { JHashState *_state =3D (JHashState *) state; @@ -3526,7 +3602,7 @@ hash_object_field_end(void *state, char *fname, bool= isnull) * Ignore nested fields. */ if (_state->lex->lex_level > 1) - return; + return JSON_SUCCESS; = /* * Ignore field names >=3D NAMEDATALEN - they can't match a record field= . @@ -3536,7 +3612,7 @@ hash_object_field_end(void *state, char *fname, bool= isnull) * has previously insisted on exact equality, so we keep this behavior.) */ if (strlen(fname) >=3D NAMEDATALEN) - return; + return JSON_SUCCESS; = hashentry =3D hash_search(_state->hash, fname, HASH_ENTER, &found); = @@ -3562,9 +3638,11 @@ hash_object_field_end(void *state, char *fname, boo= l isnull) /* must have had a scalar instead */ hashentry->val =3D _state->saved_scalar; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType hash_array_start(void *state) { JHashState *_state =3D (JHashState *) state; @@ -3573,9 +3651,11 @@ hash_array_start(void *state) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cannot call %s on an array", _state->function_name))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType hash_scalar(void *state, char *token, JsonTokenType tokentype) { JHashState *_state =3D (JHashState *) state; @@ -3591,6 +3671,8 @@ hash_scalar(void *state, char *token, JsonTokenType = tokentype) /* saved_token_type must already be set in hash_object_field_start() */ Assert(_state->saved_token_type =3D=3D tokentype); } + + return JSON_SUCCESS; } = = @@ -3840,7 +3922,7 @@ populate_recordset_worker(FunctionCallInfo fcinfo, c= onst char *funcname, PG_RETURN_NULL(); } = -static void +static JsonParseErrorType populate_recordset_object_start(void *state) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; @@ -3856,7 +3938,7 @@ populate_recordset_object_start(void *state) = /* Nested objects require no special processing */ if (lex_level > 1) - return; + return JSON_SUCCESS; = /* Object at level 1: set up a new hash table for this object */ ctl.keysize =3D NAMEDATALEN; @@ -3866,9 +3948,11 @@ populate_recordset_object_start(void *state) 100, &ctl, HASH_ELEM | HASH_STRINGS | HASH_CONTEXT); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_object_end(void *state) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; @@ -3876,7 +3960,7 @@ populate_recordset_object_end(void *state) = /* Nested objects require no special processing */ if (_state->lex->lex_level > 1) - return; + return JSON_SUCCESS; = obj.is_json =3D true; obj.val.json_hash =3D _state->json_hash; @@ -3887,9 +3971,11 @@ populate_recordset_object_end(void *state) /* Done with hash for this object */ hash_destroy(_state->json_hash); _state->json_hash =3D NULL; + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_array_element_start(void *state, bool isnull) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; @@ -3900,15 +3986,18 @@ populate_recordset_array_element_start(void *state= , bool isnull) (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("argument of %s must be an array of objects", _state->function_name))); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_array_start(void *state) { /* nothing to do */ + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_scalar(void *state, char *token, JsonTokenType tokenty= pe) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; @@ -3921,15 +4010,17 @@ populate_recordset_scalar(void *state, char *token= , JsonTokenType tokentype) = if (_state->lex->lex_level =3D=3D 2) _state->saved_scalar =3D token; + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_object_field_start(void *state, char *fname, bool isnu= ll) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; = if (_state->lex->lex_level > 2) - return; + return JSON_SUCCESS; = _state->saved_token_type =3D _state->lex->token_type; = @@ -3942,9 +4033,11 @@ populate_recordset_object_field_start(void *state, = char *fname, bool isnull) { _state->save_json_start =3D NULL; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType populate_recordset_object_field_end(void *state, char *fname, bool isnull= ) { PopulateRecordsetState *_state =3D (PopulateRecordsetState *) state; @@ -3955,7 +4048,7 @@ populate_recordset_object_field_end(void *state, cha= r *fname, bool isnull) * Ignore nested fields. */ if (_state->lex->lex_level > 2) - return; + return JSON_SUCCESS; = /* * Ignore field names >=3D NAMEDATALEN - they can't match a record field= . @@ -3965,7 +4058,7 @@ populate_recordset_object_field_end(void *state, cha= r *fname, bool isnull) * has previously insisted on exact equality, so we keep this behavior.) */ if (strlen(fname) >=3D NAMEDATALEN) - return; + return JSON_SUCCESS; = hashentry =3D hash_search(_state->json_hash, fname, HASH_ENTER, &found); = @@ -3991,6 +4084,8 @@ populate_recordset_object_field_end(void *state, cha= r *fname, bool isnull) /* must have had a scalar instead */ hashentry->val =3D _state->saved_scalar; } + + return JSON_SUCCESS; } = /* @@ -4002,39 +4097,47 @@ populate_recordset_object_field_end(void *state, c= har *fname, bool isnull) * is called. */ = -static void +static JsonParseErrorType sn_object_start(void *state) { StripnullState *_state =3D (StripnullState *) state; = appendStringInfoCharMacro(_state->strval, '{'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_object_end(void *state) { StripnullState *_state =3D (StripnullState *) state; = appendStringInfoCharMacro(_state->strval, '}'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_array_start(void *state) { StripnullState *_state =3D (StripnullState *) state; = appendStringInfoCharMacro(_state->strval, '['); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_array_end(void *state) { StripnullState *_state =3D (StripnullState *) state; = appendStringInfoCharMacro(_state->strval, ']'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_object_field_start(void *state, char *fname, bool isnull) { StripnullState *_state =3D (StripnullState *) state; @@ -4047,7 +4150,7 @@ sn_object_field_start(void *state, char *fname, bool= isnull) * object or array. The flag will be reset in the scalar action. */ _state->skip_next_null =3D true; - return; + return JSON_SUCCESS; } = if (_state->strval->data[_state->strval->len - 1] !=3D '{') @@ -4060,18 +4163,22 @@ sn_object_field_start(void *state, char *fname, bo= ol isnull) escape_json(_state->strval, fname); = appendStringInfoCharMacro(_state->strval, ':'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_array_element_start(void *state, bool isnull) { StripnullState *_state =3D (StripnullState *) state; = if (_state->strval->data[_state->strval->len - 1] !=3D '[') appendStringInfoCharMacro(_state->strval, ','); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType sn_scalar(void *state, char *token, JsonTokenType tokentype) { StripnullState *_state =3D (StripnullState *) state; @@ -4080,13 +4187,15 @@ sn_scalar(void *state, char *token, JsonTokenType = tokentype) { Assert(tokentype =3D=3D JSON_TOKEN_NULL); _state->skip_next_null =3D false; - return; + return JSON_SUCCESS; } = if (tokentype =3D=3D JSON_TOKEN_STRING) escape_json(_state->strval, token); else appendStringInfoString(_state->strval, token); + + return JSON_SUCCESS; } = /* @@ -5326,7 +5435,7 @@ iterate_json_values(text *json, uint32 flags, void *= action_state, * An auxiliary function for iterate_json_values to invoke a specified * JsonIterateStringValuesAction for specified values. */ -static void +static JsonParseErrorType iterate_values_scalar(void *state, char *token, JsonTokenType tokentype) { IterateJsonStringValuesState *_state =3D (IterateJsonStringValuesState *= ) state; @@ -5350,9 +5459,11 @@ iterate_values_scalar(void *state, char *token, Jso= nTokenType tokentype) /* do not call callback for any other token */ break; } + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType iterate_values_object_field_start(void *state, char *fname, bool isnull) { IterateJsonStringValuesState *_state =3D (IterateJsonStringValuesState *= ) state; @@ -5363,6 +5474,8 @@ iterate_values_object_field_start(void *state, char = *fname, bool isnull) = _state->action(_state->action_state, val, strlen(val)); } + + return JSON_SUCCESS; } = /* @@ -5430,7 +5543,6 @@ transform_json_string_values(text *json, void *actio= n_state, state->action_state =3D action_state; = sem->semstate =3D (void *) state; - sem->scalar =3D transform_string_values_scalar; sem->object_start =3D transform_string_values_object_start; sem->object_end =3D transform_string_values_object_end; sem->array_start =3D transform_string_values_array_start; @@ -5449,39 +5561,47 @@ transform_json_string_values(text *json, void *act= ion_state, * specified JsonTransformStringValuesAction for all values and left ever= ything * else untouched. */ -static void +static JsonParseErrorType transform_string_values_object_start(void *state) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; = appendStringInfoCharMacro(_state->strval, '{'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_object_end(void *state) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; = appendStringInfoCharMacro(_state->strval, '}'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_array_start(void *state) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; = appendStringInfoCharMacro(_state->strval, '['); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_array_end(void *state) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; = appendStringInfoCharMacro(_state->strval, ']'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_object_field_start(void *state, char *fname, bool= isnull) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; @@ -5495,18 +5615,22 @@ transform_string_values_object_field_start(void *s= tate, char *fname, bool isnull */ escape_json(_state->strval, fname); appendStringInfoCharMacro(_state->strval, ':'); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_array_element_start(void *state, bool isnull) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; = if (_state->strval->data[_state->strval->len - 1] !=3D '[') appendStringInfoCharMacro(_state->strval, ','); + + return JSON_SUCCESS; } = -static void +static JsonParseErrorType transform_string_values_scalar(void *state, char *token, JsonTokenType to= kentype) { TransformJsonStringValuesState *_state =3D (TransformJsonStringValuesSta= te *) state; @@ -5519,4 +5643,6 @@ transform_string_values_scalar(void *state, char *to= ken, JsonTokenType tokentype } else appendStringInfoString(_state->strval, token); + + return JSON_SUCCESS; } diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c index 0d37f69298..7b28a266ce 100644 --- a/src/backend/utils/fmgr/fmgr.c +++ b/src/backend/utils/fmgr/fmgr.c @@ -1614,6 +1614,51 @@ InputFunctionCallSafe(FmgrInfo *flinfo, char *str, return true; } = +/* + * Call a directly-named datatype input function, with non-exception + * handling of "soft" errors. + * + * This is like InputFunctionCallSafe, except that it is given a direct + * pointer to the C function to call. We assume that that function is + * strict. Also, the function cannot be one that needs to + * look at FmgrInfo, since there won't be any. + */ +bool +DirectInputFunctionCallSafe(PGFunction func, char *str, + Oid typioparam, int32 typmod, + fmNodePtr escontext, + Datum *result) +{ + LOCAL_FCINFO(fcinfo, 3); + + if (str =3D=3D NULL) + { + *result =3D (Datum) 0; /* just return null result */ + return true; + } + + InitFunctionCallInfoData(*fcinfo, NULL, 3, InvalidOid, escontext, NULL); + + fcinfo->args[0].value =3D CStringGetDatum(str); + fcinfo->args[0].isnull =3D false; + fcinfo->args[1].value =3D ObjectIdGetDatum(typioparam); + fcinfo->args[1].isnull =3D false; + fcinfo->args[2].value =3D Int32GetDatum(typmod); + fcinfo->args[2].isnull =3D false; + + *result =3D (*func) (fcinfo); + + /* Result value is garbage, and could be null, if an error was reported = */ + if (SOFT_ERROR_OCCURRED(escontext)) + return false; + + /* Otherwise, shouldn't get null result */ + if (fcinfo->isnull) + elog(ERROR, "input function %p returned NULL", (void *) func); + + return true; +} + /* * Call a previously-looked-up datatype output function. * diff --git a/src/bin/pg_verifybackup/parse_manifest.c b/src/bin/pg_verifyb= ackup/parse_manifest.c index 6364b01282..beff018e18 100644 --- a/src/bin/pg_verifybackup/parse_manifest.c +++ b/src/bin/pg_verifybackup/parse_manifest.c @@ -88,14 +88,14 @@ typedef struct char *manifest_checksum; } JsonManifestParseState; = -static void json_manifest_object_start(void *state); -static void json_manifest_object_end(void *state); -static void json_manifest_array_start(void *state); -static void json_manifest_array_end(void *state); -static void json_manifest_object_field_start(void *state, char *fname, - bool isnull); -static void json_manifest_scalar(void *state, char *token, - JsonTokenType tokentype); +static JsonParseErrorType json_manifest_object_start(void *state); +static JsonParseErrorType json_manifest_object_end(void *state); +static JsonParseErrorType json_manifest_array_start(void *state); +static JsonParseErrorType json_manifest_array_end(void *state); +static JsonParseErrorType json_manifest_object_field_start(void *state, c= har *fname, + bool isnull); +static JsonParseErrorType json_manifest_scalar(void *state, char *token, + JsonTokenType tokentype); static void json_manifest_finalize_file(JsonManifestParseState *parse); static void json_manifest_finalize_wal_range(JsonManifestParseState *pars= e); static void verify_manifest_checksum(JsonManifestParseState *parse, @@ -162,7 +162,7 @@ json_parse_manifest(JsonManifestParseContext *context,= char *buffer, * WAL range is also expected to be an object. If we're anywhere else in = the * document, it's an error. */ -static void +static JsonParseErrorType json_manifest_object_start(void *state) { JsonManifestParseState *parse =3D state; @@ -191,6 +191,8 @@ json_manifest_object_start(void *state) "unexpected object start"); break; } + + return JSON_SUCCESS; } = /* @@ -201,7 +203,7 @@ json_manifest_object_start(void *state) * reach the end of an object representing a particular file or WAL range= , * we must call json_manifest_finalize_file() to save the associated deta= ils. */ -static void +static JsonParseErrorType json_manifest_object_end(void *state) { JsonManifestParseState *parse =3D state; @@ -224,6 +226,8 @@ json_manifest_object_end(void *state) "unexpected object end"); break; } + + return JSON_SUCCESS; } = /* @@ -233,7 +237,7 @@ json_manifest_object_end(void *state) * should be an array. Similarly for the "WAL-Ranges" key. No other array= s * are expected. */ -static void +static JsonParseErrorType json_manifest_array_start(void *state) { JsonManifestParseState *parse =3D state; @@ -251,6 +255,8 @@ json_manifest_array_start(void *state) "unexpected array start"); break; } + + return JSON_SUCCESS; } = /* @@ -258,7 +264,7 @@ json_manifest_array_start(void *state) * * The cases here are analogous to those in json_manifest_array_start. */ -static void +static JsonParseErrorType json_manifest_array_end(void *state) { JsonManifestParseState *parse =3D state; @@ -274,12 +280,14 @@ json_manifest_array_end(void *state) "unexpected array end"); break; } + + return JSON_SUCCESS; } = /* * Invoked at the start of each object field in the JSON document. */ -static void +static JsonParseErrorType json_manifest_object_field_start(void *state, char *fname, bool isnull) { JsonManifestParseState *parse =3D state; @@ -367,6 +375,8 @@ json_manifest_object_field_start(void *state, char *fn= ame, bool isnull) "unexpected object field"); break; } + + return JSON_SUCCESS; } = /* @@ -384,7 +394,7 @@ json_manifest_object_field_start(void *state, char *fn= ame, bool isnull) * reach either the end of the object representing this file, or the end * of the manifest, as the case may be. */ -static void +static JsonParseErrorType json_manifest_scalar(void *state, char *token, JsonTokenType tokentype) { JsonManifestParseState *parse =3D state; @@ -448,6 +458,8 @@ json_manifest_scalar(void *state, char *token, JsonTok= enType tokentype) json_manifest_parse_failure(parse->context, "unexpected scalar"); break; } + + return JSON_SUCCESS; } = /* diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c index 873357aa02..83c286b89b 100644 --- a/src/common/jsonapi.c +++ b/src/common/jsonapi.c @@ -298,9 +298,9 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem) return result; = /* invoke the callback */ - (*sfunc) (sem->semstate, val, tok); + result =3D (*sfunc) (sem->semstate, val, tok); = - return JSON_SUCCESS; + return result; } = static JsonParseErrorType @@ -335,7 +335,11 @@ parse_object_field(JsonLexContext *lex, JsonSemAction= *sem) isnull =3D tok =3D=3D JSON_TOKEN_NULL; = if (ostart !=3D NULL) - (*ostart) (sem->semstate, fname, isnull); + { + result =3D (*ostart) (sem->semstate, fname, isnull); + if (result !=3D JSON_SUCCESS) + return result; + } = switch (tok) { @@ -352,7 +356,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction= *sem) return result; = if (oend !=3D NULL) - (*oend) (sem->semstate, fname, isnull); + { + result =3D (*oend) (sem->semstate, fname, isnull); + if (result !=3D JSON_SUCCESS) + return result; + } + return JSON_SUCCESS; } = @@ -373,7 +382,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem) #endif = if (ostart !=3D NULL) - (*ostart) (sem->semstate); + { + result =3D (*ostart) (sem->semstate); + if (result !=3D JSON_SUCCESS) + return result; + } = /* * Data inside an object is at a higher nesting level than the object @@ -417,7 +430,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem) lex->lex_level--; = if (oend !=3D NULL) - (*oend) (sem->semstate); + { + result =3D (*oend) (sem->semstate); + if (result !=3D JSON_SUCCESS) + return result; + } = return JSON_SUCCESS; } @@ -429,13 +446,16 @@ parse_array_element(JsonLexContext *lex, JsonSemActi= on *sem) json_aelem_action aend =3D sem->array_element_end; JsonTokenType tok =3D lex_peek(lex); JsonParseErrorType result; - bool isnull; = isnull =3D tok =3D=3D JSON_TOKEN_NULL; = if (astart !=3D NULL) - (*astart) (sem->semstate, isnull); + { + result =3D (*astart) (sem->semstate, isnull); + if (result !=3D JSON_SUCCESS) + return result; + } = /* an array element is any object, array or scalar */ switch (tok) @@ -454,7 +474,11 @@ parse_array_element(JsonLexContext *lex, JsonSemActio= n *sem) return result; = if (aend !=3D NULL) - (*aend) (sem->semstate, isnull); + { + result =3D (*aend) (sem->semstate, isnull); + if (result !=3D JSON_SUCCESS) + return result; + } = return JSON_SUCCESS; } @@ -475,7 +499,11 @@ parse_array(JsonLexContext *lex, JsonSemAction *sem) #endif = if (astart !=3D NULL) - (*astart) (sem->semstate); + { + result =3D (*astart) (sem->semstate); + if (result !=3D JSON_SUCCESS) + return result; + } = /* * Data inside an array is at a higher nesting level than the array @@ -508,7 +536,11 @@ parse_array(JsonLexContext *lex, JsonSemAction *sem) lex->lex_level--; = if (aend !=3D NULL) - (*aend) (sem->semstate); + { + result =3D (*aend) (sem->semstate); + if (result !=3D JSON_SUCCESS) + return result; + } = return JSON_SUCCESS; } @@ -1139,6 +1171,9 @@ json_errdetail(JsonParseErrorType error, JsonLexCont= ext *lex) return _("Unicode high surrogate must not follow a high surrogate."); case JSON_UNICODE_LOW_SURROGATE: return _("Unicode low surrogate must follow a high surrogate."); + case JSON_SEM_ACTION_FAILED: + /* fall through to the error code after switch */ + break; } = /* diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h index 8d31630e5c..4590ff2476 100644 --- a/src/include/common/jsonapi.h +++ b/src/include/common/jsonapi.h @@ -52,7 +52,8 @@ typedef enum JSON_UNICODE_ESCAPE_FORMAT, JSON_UNICODE_HIGH_ESCAPE, JSON_UNICODE_HIGH_SURROGATE, - JSON_UNICODE_LOW_SURROGATE + JSON_UNICODE_LOW_SURROGATE, + JSON_SEM_ACTION_FAILED /* error should already be reported */ } JsonParseErrorType; = = @@ -84,14 +85,15 @@ typedef struct JsonLexContext StringInfo strval; } JsonLexContext; = -typedef void (*json_struct_action) (void *state); -typedef void (*json_ofield_action) (void *state, char *fname, bool isnull= ); -typedef void (*json_aelem_action) (void *state, bool isnull); -typedef void (*json_scalar_action) (void *state, char *token, JsonTokenTy= pe tokentype); +typedef JsonParseErrorType (*json_struct_action) (void *state); +typedef JsonParseErrorType (*json_ofield_action) (void *state, char *fnam= e, bool isnull); +typedef JsonParseErrorType (*json_aelem_action) (void *state, bool isnull= ); +typedef JsonParseErrorType (*json_scalar_action) (void *state, char *toke= n, JsonTokenType tokentype); = = /* * Semantic Action structure for use in parsing json. + * * Any of these actions can be NULL, in which case nothing is done at tha= t * point, Likewise, semstate can be NULL. Using an all-NULL structure amo= unts * to doing a pure parse with no side-effects, and is therefore exactly @@ -100,6 +102,11 @@ typedef void (*json_scalar_action) (void *state, char= *token, JsonTokenType toke * The 'fname' and 'token' strings passed to these actions are palloc'd. * They are not free'd or used further by the parser, so the action funct= ion * is free to do what it wishes with them. + * + * All action functions return JsonParseErrorType. If the result isn't + * JSON_SUCCESS, the parse is abandoned and that error code is returned. + * If it is JSON_SEM_ACTION_FAILED, the action function is responsible + * for having reported the error in some appropriate way. */ typedef struct JsonSemAction { diff --git a/src/include/fmgr.h b/src/include/fmgr.h index b7832d0aa2..972afe3aff 100644 --- a/src/include/fmgr.h +++ b/src/include/fmgr.h @@ -704,6 +704,10 @@ extern bool InputFunctionCallSafe(FmgrInfo *flinfo, c= har *str, Oid typioparam, int32 typmod, fmNodePtr escontext, Datum *result); +extern bool DirectInputFunctionCallSafe(PGFunction func, char *str, + Oid typioparam, int32 typmod, + fmNodePtr escontext, + Datum *result); extern Datum OidInputFunctionCall(Oid functionId, char *str, Oid typioparam, int32 typmod); extern char *OutputFunctionCall(FmgrInfo *flinfo, Datum val); diff --git a/src/include/utils/jsonfuncs.h b/src/include/utils/jsonfuncs.h index 865b2ff7c1..7fad0269f6 100644 --- a/src/include/utils/jsonfuncs.h +++ b/src/include/utils/jsonfuncs.h @@ -39,11 +39,16 @@ typedef text *(*JsonTransformStringValuesAction) (void= *state, char *elem_value, /* build a JsonLexContext from a text datum */ extern JsonLexContext *makeJsonLexContext(text *json, bool need_escapes); = -/* try to parse json, and ereport(ERROR) on failure */ -extern void pg_parse_json_or_ereport(JsonLexContext *lex, JsonSemAction *= sem); +/* try to parse json, and errsave(escontext) on failure */ +extern bool pg_parse_json_or_errsave(JsonLexContext *lex, JsonSemAction *= sem, + struct Node *escontext); = -/* report an error during json lexing or parsing */ -extern void json_ereport_error(JsonParseErrorType error, JsonLexContext *= lex); +#define pg_parse_json_or_ereport(lex, sem) \ + (void) pg_parse_json_or_errsave(lex, sem, NULL) + +/* save an error during json lexing or parsing */ +extern void json_errsave_error(JsonParseErrorType error, JsonLexContext *= lex, + struct Node *escontext); = extern uint32 parse_jsonb_index_flags(Jsonb *jb); extern void iterate_jsonb_values(Jsonb *jb, uint32 flags, void *state, diff --git a/src/test/regress/expected/json.out b/src/test/regress/expecte= d/json.out index cb181226e9..af96ce4180 100644 --- a/src/test/regress/expected/json.out +++ b/src/test/regress/expected/json.out @@ -320,6 +320,25 @@ LINE 1: SELECT '{ DETAIL: Expected JSON value, but found "}". CONTEXT: JSON data, line 4: ...yveryveryveryveryveryveryveryverylongfiel= dname":} -- ERROR missing value for last field +-- test non-error-throwing input +select pg_input_is_valid('{"a":true}', 'json'); + pg_input_is_valid = +------------------- + t +(1 row) + +select pg_input_is_valid('{"a":true', 'json'); + pg_input_is_valid = +------------------- + f +(1 row) + +select pg_input_error_message('{"a":true', 'json'); + pg_input_error_message = +------------------------------------ + invalid input syntax for type json +(1 row) + --constructors -- array_to_json SELECT array_to_json(array(select 1 as a)); diff --git a/src/test/regress/expected/jsonb.out b/src/test/regress/expect= ed/jsonb.out index b2b3677482..be85676b5b 100644 --- a/src/test/regress/expected/jsonb.out +++ b/src/test/regress/expected/jsonb.out @@ -310,6 +310,31 @@ LINE 1: SELECT '{ DETAIL: Expected JSON value, but found "}". CONTEXT: JSON data, line 4: ...yveryveryveryveryveryveryveryverylongfiel= dname":} -- ERROR missing value for last field +-- test non-error-throwing input +select pg_input_is_valid('{"a":true}', 'jsonb'); + pg_input_is_valid = +------------------- + t +(1 row) + +select pg_input_is_valid('{"a":true', 'jsonb'); + pg_input_is_valid = +------------------- + f +(1 row) + +select pg_input_error_message('{"a":true', 'jsonb'); + pg_input_error_message = +------------------------------------ + invalid input syntax for type json +(1 row) + +select pg_input_error_message('{"a":1e1000000}', 'jsonb'); + pg_input_error_message = +-------------------------------- + value overflows numeric format +(1 row) + -- make sure jsonb is passed through json generators without being escape= d SELECT array_to_json(ARRAY [jsonb '{"a":1}', jsonb '{"b":[2,3]}']); array_to_json = diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql index 589e0cea36..21534ed959 100644 --- a/src/test/regress/sql/json.sql +++ b/src/test/regress/sql/json.sql @@ -81,6 +81,11 @@ SELECT '{ "averyveryveryveryveryveryveryveryveryverylongfieldname":}'::json; -- ERROR missing value for last field = +-- test non-error-throwing input +select pg_input_is_valid('{"a":true}', 'json'); +select pg_input_is_valid('{"a":true', 'json'); +select pg_input_error_message('{"a":true', 'json'); + --constructors -- array_to_json = diff --git a/src/test/regress/sql/jsonb.sql b/src/test/regress/sql/jsonb.s= ql index 8d25966267..bc44ad1518 100644 --- a/src/test/regress/sql/jsonb.sql +++ b/src/test/regress/sql/jsonb.sql @@ -86,6 +86,12 @@ SELECT '{ "averyveryveryveryveryveryveryveryveryverylongfieldname":}'::jsonb; -- ERROR missing value for last field = +-- test non-error-throwing input +select pg_input_is_valid('{"a":true}', 'jsonb'); +select pg_input_is_valid('{"a":true', 'jsonb'); +select pg_input_error_message('{"a":true', 'jsonb'); +select pg_input_error_message('{"a":1e1000000}', 'jsonb'); + -- make sure jsonb is passed through json generators without being escape= d SELECT array_to_json(ARRAY [jsonb '{"a":1}', jsonb '{"b":[2,3]}']); = ------- =_aaaaaaaaaa0--